Add optional L2 normalization for Euclidean metric in kNN widget

[gmolledaj]

What's your use case?
I am using the kNN widget in Orange for text classification tasks based on document embeddings (e.g., from the Document Embedding widget).

In this context, similarity between instances is typically measured using cosine similarity. However, since cosine distance is not available in the kNN widget, a common workaround is to use Euclidean distance on L2-normalized vectors, which is mathematically equivalent in terms of neighbor ranking. Source Bouhsine, T. (2026, febrero 23). In Defense of Cosine Similarity: Normalization Eliminates the Gauge Freedom: "3. Euclidean k-NN on normalized embeddings = cosine search. By Corollary 5, FAISS flat L2 search on unit-normalized vectors produces the same neighbor rankings as cosine search. Use whichever is faster for your infrastructure."

Currently, Orange does not provide an easy way to apply L2 normalization per instance within the standard workflow. While it is possible to implement this via a Python Script widget, this introduces additional complexity and breaks the simplicity and reproducibility of the pipeline, especially for non-technical users.

Moreover, applying normalization externally via the Preprocess widget overrides the learner’s internal preprocessing steps, which can lead to unintended side effects.

What's your proposed solution?
Add an optional checkbox in the kNN widget:

Normalize instances (L2)

This option should:

apply L2 normalization per instance before distance computation
be enabled only when the Euclidean metric is selected
be integrated into the learner’s internal preprocessing pipeline (without overriding existing preprocessing steps)

This would allow users to effectively use cosine-equivalent similarity while preserving the current behavior and optimizations of the kNN implementation.

Are there any alternative solutions?
Yes, but they are less ideal:

1.- Using a Python Script widget
· requires custom code
· reduces usability for non-programmers
· makes workflows less transparent
2.- Adding L2 normalization in the Preprocess widget
· applies globally rather than model-specific
· overrides the learner’s internal preprocessing
· may affect other models unintentionally
3.- Adding cosine distance directly to kNN
· would require switching to brute-force search in many cases
· may negatively impact performance
· adds complexity to the implementation

For these reasons, integrating L2 normalization as an optional feature within the kNN widget seems to be the most practical and user-friendly solution.

[gmolledaj]

I have implemented a prototype of the proposed solution and attached the corresponding diffs for review:

· kNN widget modification
· associated unit tests

Summary of the implementation

The changes introduce an optional checkbox:

Normalize instances (L2)

with the following behavior:

· Applies L2 normalization per instance before distance computation
· Only enabled when the Euclidean metric is selected
· Automatically disabled (and unchecked) when switching to other metrics
· Integrated into the learner’s preprocessing pipeline without overriding existing preprocessors
· Safely handles cases where no external preprocessors are provided (NoneType fix)

Additional notes
· The implementation preserves the current behavior of the kNN widget unless the option is explicitly enabled
· It does not introduce changes to other widgets or global preprocessing logic
· Tests have been updated to cover:

• activation/deactivation of the option
• correct inclusion of the L2 preprocessor
• behavior when no external preprocessors are present

Please let me know if you would prefer this feature to be implemented differently (e.g., as a general preprocessing step instead of being tied to kNN).

Availability

Due to current commitments, I likely won’t be able to carry this forward into a full PR before the summer. If someone from the team or the community would like to pick this up and integrate it properly, I’d be happy to assist or clarify any part of the implementation.

changes_owknn_and_test_diff.zip

[markotoplak]

In spectroscopy people tend to have a similar need, and there they have such options in "Preprocess spectra". So you motivation was text data? I am surprised that there is no similar preprocessing option there. @ajdapretnar, is this true?

I am not against this and it is an interesting way to introduce cosine distance into k-means, but why not just name it cosine distance then? We need to discuss this internally.

[gmolledaj]

Yes, my motivation came mainly from document embeddings generated by the Text add-on (Document Embedding widget), where cosine similarity is the standard choice. One of my use cases is teaching sentiment analysis on tourism reviews, where students work with document embeddings and kNN classification in Orange.

My intention was not to implement cosine distance itself, but to expose the common workflow of L2-normalizing the instances before applying Euclidean kNN. On unit-normalized embeddings, Euclidean kNN produces exactly the same neighbour ranking as cosine kNN, while preserving the current Euclidean implementation and its optimizations.

Since Orange is also widely used for teaching, I actually see some educational value in keeping this option explicit. It makes it clear to users that the workflow is "L2 normalization + Euclidean distance", helping them understand why this is equivalent to cosine neighbour ranking on normalized embeddings.

That said, I would also be perfectly happy if the project prefers introducing a native Cosine metric instead. My main goal is simply to make this workflow easily accessible without requiring users to resort to Python Script or external preprocessing.