Index: spec/filter-spec.coffee IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- spec/filter-spec.coffee (revision 76fb30a5fbde17420822e069412c7b5e94030d88) +++ spec/filter-spec.coffee (revision ) @@ -423,10 +423,18 @@ expect(bestMatch(candidates, 'CCCa')).toBe candidates[0] expect(bestMatch(candidates, 'ccca')).toBe candidates[1] + it "prefer CamelCase to start-of-word, when length are reasonably similar", -> + candidates = [ + 'JSON', + 'JavaScript' + ] + expect(bestMatch(candidates, 'JS', debug:true)).toBe candidates[1] + + #--------------------------------------------------- # # Path / Fuzzy finder @@ -663,6 +671,8 @@ ] expect(bestMatch(candidates, 'appcon')).toBe candidates[1] + #expect(bestMatch(candidates, path.join('con','appcon'), debug:true )).toBe candidates[1] + expect(bestMatch(candidates, 'con appcon')).toBe candidates[1] it "prefer path together to shorter path", -> Index: src/scorer.coffee IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- src/scorer.coffee (revision 76fb30a5fbde17420822e069412c7b5e94030d88) +++ src/scorer.coffee (revision ) @@ -13,6 +13,7 @@ # Base point for a single character match # This balance making patterns VS position and size penalty. wm = 150 +acronym_bonus = 10 #Fading function pos_bonus = 20 # The character from 0..pos_bonus receive a greater bonus for being at the start of string. @@ -311,7 +312,7 @@ # match IS a word boundary # choose between taking part of consecutive characters or consecutive acronym if start - return posBonus + wm * ( (if acro_score > csc_score then acro_score else csc_score) + 10 ) + return posBonus + wm * ( (if acro_score > csc_score then acro_score else csc_score) + acronym_bonus ) # normal Match return posBonus + wm * csc_score @@ -442,8 +443,8 @@ basenameScore = (subject, subject_lw, prepQuery, fullPathScore) -> return 0 if fullPathScore is 0 + basePathScore = 0 - # Skip trailing slashes end = subject.length - 1 end-- while subject[end] is PathSeparator @@ -452,20 +453,26 @@ basePos = subject.lastIndexOf(PathSeparator, end) #If no PathSeparator, no base path exist. - return fullPathScore if (basePos is -1) + if (basePos is -1) + basePathScore = fullPathScore + else + - # Get the number of folder in query - depth = prepQuery.depth + # Get the number of folder in query + depth = prepQuery.depth - # Get that many folder from subject - while(depth-- > 0) - basePos = subject.lastIndexOf(PathSeparator, basePos - 1) + # Get that many folder from subject + while(depth-- > 0) + basePos = subject.lastIndexOf(PathSeparator, basePos - 1) - if (basePos is -1) then return fullPathScore #consumed whole subject ? + if (basePos is -1) #consumed whole subject ? + basePathScore = fullPathScore + break + if basePathScore is 0 - # Get basePath score - basePos++ - end++ - basePathScore = doScore(subject[basePos...end], subject_lw[basePos...end], prepQuery) + # Get basePath score + basePos++ + end++ + basePathScore = doScore(subject[basePos...end], subject_lw[basePos...end], prepQuery) # Final score is linear interpolation between base score and full path score. # For low directory depth, interpolation favor base Path then include more of full path as depth increase \ No newline at end of file Index: README.md IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- README.md (revision 76fb30a5fbde17420822e069412c7b5e94030d88) +++ README.md (revision ) @@ -174,7 +174,7 @@ - **A:** First, once you have matched everything, there's no danger of splitting the query. Then, that bonus exists to ensure exact matches will bubble up in the firsts results, despite longer/deeper path. If, after more test and tuning, we realize it's not needed, we'll be happy to remove it, the fewer corner cases, the merrier. - **Q:** Why are you using lowercase to detect CamelCase? - - **A** CamelCase are detected as a switch from lowercase to UPPERCASE. Defining UPPERCASE as not-lowercase, allow case-invariants characters to count as lowercase. For example `Git Push` the `P` of push will be recognised as CamelCase because we consider `` as lowercase. + - **A** CamelCase are detected as a switch from lowercase to UPPERCASE. Defining UPPERCASE as not-lowercase, allow case-invariants characters to count as lowercase. For example `Git Push` the `P` of push will be recognised as CamelCase because we consider `` as lowercase. ### 4. Tertiary score attributes are subject size, match position and directory depth @@ -191,7 +191,7 @@ - The maximum range is `score(query,query)` whatever that number is. A longer query will have a greater maximal score. -- Score exist mainly to for relative order with other scores of the same query and to implements scoring rule described above. +- Score exist mainly to for relative order with other scores of the same query and to implements scoring rule described above. - Score have a high dynamic range and consider a lot of information. Equality is unlikely. For that reason, **multiplicative bonuses should be preferred over additive ones**. @@ -250,7 +250,7 @@ ### Path Scoring -- Score for a given path is computed from the score of the fullpath and score of the filename. For low directory depth, the influence of both is about equal. But, for deeper directory, there is less retrieval effect (importance of basename) +- Score for a given path is computed from the score of the fullpath and score of the filename. For low directory depth, the influence of both is about equal. But, for deeper directory, there is less retrieval effect (importance of basename) - The full path is penalized twice for size. Once for its own size, then a second time for the size of the basename. Extra basename penalty is dampened a bit. @@ -342,7 +342,7 @@ LCS is not immediately useful for fuzzaldrin needs. Because fuzzaldrin require ALL characters of the query to be in subject to have a score greater than 0, LCS for all positive candidates would be the length of the query. -However, the dynamic programming table used to solve LCS is very useful to our need. The ability to select the best path and skip that `g` even if it is present in both query and candidate is the key to improves over left-most alignment. All we need for this to works is a bit more detail in score than 0 or 1. +However, the dynamic programming table used to solve LCS is very useful to our need. The ability to select the best path and skip that `g` even if it is present in both query and candidate is the key to improves over left-most alignment. All we need for this to works is a bit more detail in score than 0 or 1. ### Similarity score @@ -376,9 +376,9 @@ diag = 25 row = 30, 31, 32, 33, 34, **35**, 26, 27, 28, 29 -To compute value of cell [3,6] we take +To compute value of cell [3,6] we take - UP value (26) from the row. -- DIAG value, from the diag register. +- DIAG value, from the diag register. - LEFT value from the previously computed value: 35 ### Initial values @@ -391,7 +391,7 @@ We set up the row vector with the size of the query. Using a full matrix, scoring a query of size 5 against a path of size 100, would require a 500 cells. Instead, we use a 5 item row + some registers. This should ease memory management pressure. -Each character of the query manages its best score. More precisely, each cell `row[j]` manage the best score so far of matching `query[0..j]` against candidate[0..i]. +Each character of the query manages its best score. More precisely, each cell `row[j]` manage the best score so far of matching `query[0..j]` against candidate[0..i]. ### Consecutive Score (Neighbourhood) Matrix. @@ -411,11 +411,11 @@ - Of those 5, 1 is a exact case-sensitive match. - That particular user almost always wants that case sensitive match. -Should we optimize for case sensitive `indexOf` before trying other things? Our answer to that question is no. +Should we optimize for case sensitive `indexOf` before trying other things? Our answer to that question is no. Case sensitive exact match are valuable because they are rare. Even if the user tries to get them, for each one of those we have to reject 995 entry and deal with 4 other kinds of matches. -This is our first principle for optimization: **Most of the haystack is not the needle**. Because rejection of candidate happens often, we should be very good at doing that. +This is our first principle for optimization: **Most of the haystack is not the needle**. Because rejection of candidate happens often, we should be very good at doing that. Failing a test for case-sensitive `indexOf` tell us exactly nothing for case-insensitive `indexOf`, or acronyms, or even scattered letters. That test is too specific. To reject match efficiently, we should aim for the lowest common denominator: scattered case-insensitive match. @@ -424,7 +424,7 @@ ### Most of the haystack is not the needle -We just have shown how that sentence applies at the candidate level, but it is also at the character level. +We just have shown how that sentence applies at the candidate level, but it is also at the character level. **Let's consider this line: `if (subject_lw[i] == query_lw[j])`** This test is for match points (or hits). It refers to the `diag+1` in the algorithm description, with the `+1` being refined to handle the differents levels of character and neighborhood similarity. @@ -434,7 +434,7 @@ Let's consider an alphabet that contain 26 lowercase letters, 10 numbers, a few symbols ` _!?=<>`. That is a 40+ symbol alphabet. Under a uniform usage model of those symbols, we have the hit condition occurs about 2.5% of the time (1/40). If we suppose only 10-20 of those characters are popular, the hit rate is about 5-10%. -This means we'll try to minimize the number of operation that happens outside of math points. In that context, increasing the cost of a hit, while decreasing the cost of non-hits looks like a possibly worthwhile proposition. +This means we'll try to minimize the number of operation that happens outside of math points. In that context, increasing the cost of a hit, while decreasing the cost of non-hits looks like a possibly worthwhile proposition. A canonical example of this is that, instead of testing each character against the list of separators, setting a flag for next character being a start-of-word, we first confirm a match then look behind for separator. This characterization work is sometimes repeated more than once, but so far this scheme benchmarked better than alternatives we have tried to avoid doing extra work. @@ -449,11 +449,11 @@ - Search is carried as user type (the query is not intentional) - The intentional query is not fully typed, match-all is a temporary step. -One way to deal with that is not to use the full matching algorithm when we can deal with something simpler. This is what we have done while searching for `indexOf` instance. +One way to deal with that is not to use the full matching algorithm when we can deal with something simpler. This is what we have done while searching for `indexOf` instance. One special note: Acronym still have to be checked even if we have an exact match: for example query `su` against `StatusUrl`. As an exact match it is poor: 'Statu**sU**rl' is a middle of word match and have the wrong case. However as an acronym it is great: '**S**tatus**U**rl'. That motivated us to create the specialized `scoreAcronyms`. -What is nice is that while `scoreAcronyms` was created to speed up exact matches search, it also provided very valuable information for accuracy. It later became a corner stone in the processing of accidental acronym. +What is nice is that while `scoreAcronyms` was created to speed up exact matches search, it also provided very valuable information for accuracy. It later became a corner stone in the processing of accidental acronym. The result is that for exact matches and exact acronym matches we bypass the optimal alignment algorithm, giving very fast results. We still have to deal with fuzzier stacks of needles and the next two optimization address this. Index: spec/filter-debug-spec.coffee IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== --- spec/filter-debug-spec.coffee (revision ) +++ spec/filter-debug-spec.coffee (revision ) @@ -0,0 +1,1 @@ + \ No newline at end of file