False negative: Doesn't flag "show case" to become "showcase" in some real-word contextx

[hippietrail]

I noticed a comment on another GitHub project that used "show case" instead of "showcase" but Harper doesn't flag it. I was able to craft another context in which Harper does flag it.

Not the green dotted underline is Chrome flagging this. It's also worth noting that this context does not appear to be native English so it's hard to guess how common uses similar enough to this one are. Need to check if a naïve implementation will result in false positives.

[JeetKarena]

1. POS (Part-of-Speech) Tagging Ambiguity: Both "show" and "case" are valid individual words (verbs/nouns). In certain sentence structures, a parser might misidentify "show case" as a legitimate verb-noun or noun-noun sequence (e.g., "to show [a] case" or "show case studies").
2. Strict Rule Guards to Avoid False Positives: To prevent flagging valid phrases like "In that TV show, case files were analyzed...", the rule for combining "show case" $\rightarrow$ "showcase" likely relies on strict surrounding context (e.g., specific POS tags or syntactic structures). If that context isn't matched, the rule safely defaults to doing nothing.
3. Non-Native Phrasing & Parser Confusion: Since the context comes from non-native English usage, the sentence structure might create an unexpected Abstract Syntax Tree (AST) or invalid POS tags. If the parser gets confused by surrounding grammatical errors, downstream pattern-matching rules for compound words often fail to trigger.

A simple rule check on the token sequence ["show", "case"] with a guard against preceding/following nouns (like "studies" or "files") might catch these without introducing too many false positives.

[hippietrail]

2. Strict Rule Guards to Avoid False Positives: To prevent flagging valid phrases like "In that TV show, case files were analyzed...", the rule for combining "show case"
   →
   "showcase" likely relies on strict surrounding context (e.g., specific POS tags or syntactic structures). If that context isn't matched, the rule safely defaults to doing nothing.

This smells like an AI observation. I know in my own AI usage with Harper that they always assume punctuation between words is invisible to our linters. A linter would have to do to things explicitly to match "show, case". 1) change from Chunk-based to Sentence-based and 2) specify the comma as well as the whitespace between the two word tokens, or abuse .any/.anything methods.

3. Non-Native Phrasing & Parser Confusion: Since the context comes from non-native English usage, the sentence structure might create an unexpected Abstract Syntax Tree (AST) or invalid POS tags. If the parser gets confused by surrounding grammatical errors, downstream pattern-matching rules for compound words often fail to trigger.

So far Harper doesn't build ASTs, for this very reason. (Probably not the only reason.) But I suspect that at some point some kind of mini-ASTs might be implemented. Harper currently doesn't have an exposed pos-tagging step and linters use the lexical POSes specified in the dictionary (and annotations.json). I believe there might be a pos tagger behind the UPOS stuff but I never grokked that stuff well.

A simple rule check on the token sequence ["show", "case"] with a guard against preceding/following nouns (like "studies" or "files") might catch these without introducing too many false positives.

I've been working on a tool that leverages Google Ngram Viewer to compare the top collocations of variants / confusable terms:

cngd % cargo run --release show\ case showcase
ℹ️ URL: https://books.google.com/ngrams/graph?content=*+showcase%2Cshowcase+*%2C*+show+case%2Cshow+case+*
I | that|will «« showcase »» a|his|the|their|your | D/I
 | or|your «« show case »» at|is|or|was|with | N

cngd % cargo run --release show\ cases showcases
ℹ️ URL: https://books.google.com/ngrams/graph?content=*+showcases%2Cshowcases+*%2C*+show+cases%2Cshow+cases+*
I | also|it|that|which «« showcases »» a|her|his|some|the | I/D
N | for|new|or|to «« show cases »» on|or|were|where|with | N

The format is: POSes that occur only before this variant | words that occur only before this variant | this variant | words that only occur after this variant | POSes that occur only after this variant.

[JeetKarena]

This smells like an AI observation. I know in my own AI usage with Harper that they always assume punctuation between words is invisible to our linters. A linter would have to do to things explicitly to match "show, case". 1) change from Chunk-based to Sentence-based and 2) specify the comma as well as the whitespace between the two word tokens, or abuse .any/.anything methods.

yup that is ai observation/Refactoring. if it not build ATS

So far Harper doesn't build ASTs, for this very reason. (Probably not the only reason.) But I suspect that at some point some kind of mini-ASTs might be implemented. Harper currently doesn't have an exposed pos-tagging step and linters use the lexical POSes specified in the dictionary (and annotations.json). I believe there might be a pos tagger behind the UPOS stuff but I never grokked that stuff well.

group tokens into noun phrases / verb phrases without building a full recursive tree. This is the realistic middle ground: cheap enough to keep Harper fast, but gives rules a notion of "is this + case part of the same NP" instead of raw adjacency. Sounds like this is what you mean by "mini-ASTs" — and it's the standard answer to this exact problem in shallow NLP pipelines (chunking/shallow parsing exists precisely because full parsing is expensive and often overkill for this class of task).

I've been working on a tool that leverages Google Ngram Viewer to compare the top collocations of variants / confusable terms

pretty Good

[hippietrail]

So far Harper doesn't build ASTs, for this very reason. (Probably not the only reason.) But I suspect that at some point some kind of mini-ASTs might be implemented. Harper currently doesn't have an exposed pos-tagging step and linters use the lexical POSes specified in the dictionary (and annotations.json). I believe there might be a pos tagger behind the UPOS stuff but I never grokked that stuff well.

group tokens into noun phrases / verb phrases without building a full recursive tree. This is the realistic middle ground: cheap enough to keep Harper fast, but gives rules a notion of "is this + case part of the same NP" instead of raw adjacency. Sounds like this is what you mean by "mini-ASTs" — and it's the standard answer to this exact problem in shallow NLP pipelines (chunking/shallow parsing exists precisely because full parsing is expensive and often overkill for this class of task).

Even this is harder than you imagine. I've had several attempts. I don't have deep experience with production NLP stuff though but I've played with my own stuff for a few decades and have some linguistic knowledge at a hobby level. Basically a very large percentage of words are valid as two or more POS, sometimes it's easy to see both, other times you can only conceive of one until you consult a dictionary. This is also true for individual properties POSes like singular vs plural, countable vs mass, irregular verbs where one form serves two tenses, or words that are one tense of one verb and another tense of another verb.

If you do have some experience with that stuff it would be awesome to see. Also if you can manage to get a coding agent to produce something workable.

One of the biggest problems with coding agents for Harper is that for tests to be of any real use we need to test against real-word text, and not test made up based on assumptions, by either humans or AIs. I collect them using Google but it seems most/all agents don't have access to something like Googling, probably due to robots.txt and licensing, but maybe Google's agents have tools/skills for full Googling. If so, maybe the agents lack the imagination to think up ways to Google for examples using wildcards and OR operators, or maybe they have to be shown how and can't spontaneously think of how to do it on their own.

But I'm very interested if you can get an AI around those problems too.

[JeetKarena]

trying won't hurt silicon valley :)

[JeetKarena]

experimenting with it will let you know if find anything handy ^_~