This article provides yet another example of how even simple changes can have noticeable impact on highly optimized production systems. The hero of the story is going to be
The code clearly doesn’t assume that all vectors have the same length, so it uses a separate dividers vector to keep track of the number of summed elements. Golang, very fortunately, doesn’t like implicit conversions, so both division operands have to have the same type, in this case float32. Because of this, it seems reasonable that dividers uses []float32 type. But is this the best we can do? Well, floating point arithmetic is known for its complexity and certainly comes at a cost, so what if we try using int32 instead? Let’s give it a try
Note that we now have a int32 to float32 conversion to make division operator happy, but what about performance?
suggests that we’re getting ~15% speed boost even on such a small example
vs
This shouldn’t be all that surprising given that float32 increment translates into
whereas int32 increment is just a single INCL instruction
Just don’t forget that those int32 to float32 conversions are not free
so this optimization is an improvement only if there is a large enough number of increments to pay for these conversions. Fortunately that’s the case for any interesting vectors input.
In the next article we’ll continue improving Weaviate database, so don’t forget to subscribe to not miss it.