import { language, languageConfiguration } from '../expr'; import { validateExpr } from '../validation'; import { getExprCompletionProvider } from '../completion/getCompletionProvider'; describe('expr language definition', () => { it('should have correct language id prefix', () => { expect(language.tokenPostfix).toBe('.expr'); }); it('should define keywords', () => { expect(language.keywords).toContain('let'); expect(language.keywords).toContain('true'); expect(language.keywords).toContain('false'); expect(language.keywords).toContain('nil'); expect(language.keywords).toContain('in'); expect(language.keywords).toContain('not'); expect(language.keywords).toContain('and'); expect(language.keywords).toContain('or'); expect(language.keywords).toContain('if'); expect(language.keywords).toContain('else'); }); it('should define builtin functions', () => { expect(language.builtinFunctions).toContain('len'); expect(language.builtinFunctions).toContain('now'); expect(language.builtinFunctions).toContain('filter'); expect(language.builtinFunctions).toContain('map'); expect(language.builtinFunctions).toContain('all'); expect(language.builtinFunctions).toContain('type'); expect(language.builtinFunctions).toContain('bitand'); }); it('should define operators', () => { expect(language.operators).toContain('+'); expect(language.operators).toContain('-'); expect(language.operators).toContain('*'); expect(language.operators).toContain('**'); expect(language.operators).toContain('..'); expect(language.operators).toContain('|'); expect(language.operators).toContain('?.'); expect(language.operators).toContain('??'); expect(language.operators).toContain('matches'); expect(language.operators).toContain('in'); }); it('should have comment configuration', () => { expect(languageConfiguration.comments.lineComment).toBe('//'); expect(languageConfiguration.comments.blockComment).toEqual(['/*', '*/']); }); it('should have bracket configuration', () => { expect(languageConfiguration.brackets).toContainEqual(['{', '}']); expect(languageConfiguration.brackets).toContainEqual(['[', ']']); expect(languageConfiguration.brackets).toContainEqual(['(', ')']); }); it('should have auto closing pairs', () => { expect(languageConfiguration.autoClosingPairs).toContainEqual({ open: '"', close: '"' }); expect(languageConfiguration.autoClosingPairs).toContainEqual({ open: "'", close: "'" }); expect(languageConfiguration.autoClosingPairs).toContainEqual({ open: '`', close: '`' }); expect(languageConfiguration.autoClosingPairs).toContainEqual({ open: '(', close: ')' }); }); it('should have tokenizer with root section', () => { expect(language.tokenizer).toHaveProperty('root'); expect(Array.isArray(language.tokenizer.root)).toBe(true); }); it('should have all required tokenizer states', () => { const states = Object.keys(language.tokenizer); expect(states).toContain('root'); expect(states).toContain('whitespace'); expect(states).toContain('comments'); expect(states).toContain('comment'); expect(states).toContain('numbers'); expect(states).toContain('strings'); expect(states).toContain('string_double'); expect(states).toContain('string_single'); expect(states).toContain('string_backtick'); expect(states).toContain('bytes'); expect(states).toContain('bytes_double'); expect(states).toContain('bytes_single'); }); }); describe('expr validation', () => { it('should return empty markers for empty string', () => { const markers = validateExpr(''); expect(markers).toHaveLength(0); }); it('should return empty markers for valid expression', () => { const markers = validateExpr('let x = 42; x * 2'); expect(markers).toHaveLength(0); }); it('should detect unclosed single quote', () => { const markers = validateExpr("let x = 'hello"); const quoteMarkers = markers.filter((m) => (m.message || '').includes('quote')); expect(quoteMarkers.length).toBeGreaterThan(0); }); it('should detect unclosed double quote', () => { const markers = validateExpr('let x = "hello'); const quoteMarkers = markers.filter((m) => (m.message || '').includes('quote')); expect(quoteMarkers.length).toBeGreaterThan(0); }); it('should detect unclosed backtick', () => { const markers = validateExpr('let x = `hello'); const quoteMarkers = markers.filter((m) => (m.message || '').includes('quote') || (m.message || '').includes('backtick')); expect(quoteMarkers.length).toBeGreaterThan(0); }); it('should detect unmatched opening parenthesis', () => { const markers = validateExpr('let x = (1 + 2'); const parenMarkers = markers.filter((m) => (m.message || '').includes('parenthesis') || (m.message || '').includes('end of expression') || (m.message || '').includes('")"')); expect(parenMarkers.length).toBeGreaterThan(0); }); it('should detect mismatch', () => { const markers = validateExpr('let x = 1 + 2)'); expect(markers.length).toBeGreaterThan(0); }); it('should detect unmatched opening bracket', () => { const markers = validateExpr('let x = [1, 2'); const bracketMarkers = markers.filter((m) => (m.message || '').includes('bracket') || (m.message || '').includes('end of expression') || (m.message || '').includes('"]"')); expect(bracketMarkers.length).toBeGreaterThan(0); }); it('should detect unmatched closing bracket', () => { const markers = validateExpr('let x = 1, 2]'); expect(markers.length).toBeGreaterThan(0); }); it('should detect unmatched opening curly brace', () => { const markers = validateExpr('let x = {a: 1'); expect(markers.length).toBeGreaterThan(0); }); it('should detect unmatched closing curly brace', () => { const markers = validateExpr('let x = a: 1}'); expect(markers.length).toBeGreaterThan(0); }); it('should not flag valid closed quotes', () => { const markers = validateExpr('let name = "hello"'); const quoteMarkers = markers.filter((m) => (m.message || '').includes('quote')); expect(quoteMarkers).toHaveLength(0); }); it('should not flag valid closed parentheses', () => { const markers = validateExpr('let x = (1 + 2) * 3'); expect(markers).toHaveLength(0); }); it('should handle block comments', () => { const markers = validateExpr('let x = 1 /* this is a comment */ + 2'); expect(markers).toHaveLength(0); }); it('should handle multi-line block comments', () => { const markers = validateExpr('/* start\nmiddle\nend */ let x = 1'); expect(markers).toHaveLength(0); }); it('should handle balanced quotes', () => { const markers = validateExpr(`let name = "hello" + 'world'`); const quoteMarkers = markers.filter((m) => (m.message || '').includes('quote')); expect(quoteMarkers).toHaveLength(0); }); }); describe('expr parser syntax validation', () => { it('should accept valid expressions', () => { const validExprs = [ '42', '"hello"', 'true', 'nil', 'a + b', 'a * b + c', 'a > 5', 'a || b && c', 'user.Name', 'user?.Name', 'arr[0]', 'arr[1:3]', 'foo(1, 2)', 'len(arr)', 'filter(arr, {# > 2})', '[1, 2, 3]', '{a: 1, b: 2}', 'let x = 42; x * 2', 'a ? b : c', 'a ?: c', 'a ?? b', 'a | upper()', 'a matches "foo"', 'a contains "bar"', 'now()', 'a + b; c + d', ]; for (const expr of validExprs) { const markers = validateExpr(expr); const syntaxErrors = markers.filter((m) => m.severity === 8); expect(syntaxErrors.map((e) => `${expr}: ${e.message}`)).toEqual([]); } }); it('should accept expressions from Nightingale calc tests', () => { const calcExprs = [ '一个 + $.B - $.C', '($A.err_count >0&& $A.err_count <=3)||($B.err_count>0 && $B.err_count <=5)', '$.C - $.D + $.A', '$.B / $.C * $.D', '$.A * $.B + $.C', '$.D - $.A / $.B', '$.C + $.D - $.A', '$.B * $.A - $.D', '$.A / $.B + $.C', '$.D + $.A * $.B', '($A / $B) + ($C * $D)', '($.A - $.B) / ($.C + $.D)', '($.A + $.B) * ($.C - $.D)', '($.A * $.B) / ($.C - $.D)', '$.A + ($.B * $.C) / $.D', '($.A + $.B) - ($.C * $.D)', '$.A / ($.B - $.C) * $.D', '($.A - $.B) * ($.C / $.D)', '$.A/$.B*$.D', '$.A/$.B*$.C', '$.A/($.B*$.C)', '$.A + $.B', '$.A - $.B', '$.A * $.B', '$.A / $.B', '($.A + $.B) / ($.C - $.D)', '$.A > $.B', '$A.yesterday_rate > 0.1 && $A.last_week_rate>0.1 or ($A.今天 >300 || $A.昨天>300 || $A.上周今天 > 300)', '$A.count > 0', '$A.todayRate<0.3 && $A.yesterdayRate<0.3 && $A.lastweekRate<0.3', '$A.todayRate<0.1 && $A.yesterdayRate<0.1 && $A.lastweekRate<0.1', '$A.agent == 11 && $A.todayRate<0.3 && $A.yesterdayRate<0.3 && $A.lastweekRate<0.3', '$A<0.1 && $A.yesterdayRate<0.1 && $A.lastweekRate<0.1', '$A.today_rate<0.1 && $A.yesterday_rate<0.1 && $A.lastweek_rate<0.1', '$B.today_rate<0.1 && $A.yesterday_rate<0.1 && $A.lastweek_rate<0.1', '($A.yesterday_rate > 2 && $A.byesterday_rate > 2) or ($A.yesterday_rate <= 0.7 && $A.byesterday_rate <= 0.7)', '($A.yesterday_rate > 1.5 && $A.byesterday_rate > 1.5) or ($A.yesterday_rate <= 0.8 && $A.byesterday_rate <= 0.8)', '($A.yesterday_rate > 1.0 && $A.byesterday_rate > 1.0 ) or ($A.yesterday_rate <= 0.9 && $A.byesterday_rate <= 0.9)', '$A.count > 100 or $A.count2 > -3', '$.A < $.B/$.B*4', '$.A >= $.B', '$.A <= $.B', '$.A != $.B', '$.A + $.B > $.C', '$.A - $.B < $.C', '$.A * $.B > $.C', '$.A / $.B*$.C < $.C', '($.A + $.B) > $.C && $.A >0', '($.A + $.B) > $.C || $.A < 0', '($.A + $.B) * $.C < $.D', '($.A + ($.B - $.C)) * $.D > $.E', ' ( true || false ) && true', '$.A in ["admin", "moderator"]', '$.A not in [1, 2, 3]', '$.A contains $.B', '$.A not contains $.B', '$.A matches $.B', 'between($.A, [100,200])', 'not between($.A, [100.3,200.3])', ]; for (const expr of calcExprs) { const markers = validateExpr(expr); const syntaxErrors = markers.filter((m) => m.severity === 8); expect(syntaxErrors.map((e) => `${expr}: ${e.message}`)).toEqual([]); } }); it('should report unexpected token', () => { const markers = validateExpr('42 @ 10'); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); expect(errors[0].message).toContain('unexpected token'); }); it('should report unexpected end of expression', () => { const markers = validateExpr('foo('); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); }); it('should report expected identifier after let', () => { const markers = validateExpr('let 42 = 10'); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); expect(errors[0].message).toContain('variable name'); }); it('should report expected = after variable name', () => { const markers = validateExpr('let x 10'); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); expect(errors[0].message).toContain('expected "="'); }); it('should report missing map value after colon', () => { const markers = validateExpr('{a:}'); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); }); it('should report unexpected token in array', () => { const markers = validateExpr('[1 2]'); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); }); it('should report missing closing parenthesis', () => { const markers = validateExpr('(1 + 2'); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); }); it('should report missing property after dot', () => { const markers = validateExpr('user.'); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); }); it('should report missing identifier after pipe', () => { const markers = validateExpr('a |'); const errors = markers.filter((m) => m.severity === 8); expect(errors.length).toBeGreaterThan(0); }); it('should handle complex valid expressions', () => { const markers = validateExpr('filter(users, {.Age > 18 && .Name startsWith "J"})'); const errors = markers.filter((m) => m.severity === 8); expect(errors).toHaveLength(0); }); it('should validate expression with pipe and builtin', () => { const markers = validateExpr('user.Name | upper() | split(" ")'); const errors = markers.filter((m) => m.severity === 8); expect(errors).toHaveLength(0); }); }); describe('expr completion provider', () => { it('should return a completion provider', () => { const provider = getExprCompletionProvider(); expect(provider).toBeDefined(); expect(typeof provider.provideCompletionItems).toBe('function'); }); it('should provide keywords in suggestions', async () => { const provider = getExprCompletionProvider(); const mockModel: any = { getWordUntilPosition: () => ({ startColumn: 1, endColumn: 1, word: '' }), getValue: () => '', getLineContent: () => '', }; const mockPosition: any = { lineNumber: 1, column: 1 }; const emptyContext: any = {}; const emptyToken: any = {}; const result = await provider.provideCompletionItems(mockModel, mockPosition, emptyContext, emptyToken); const suggestions = (result && result.suggestions) || []; const keywordLabels = suggestions.filter((s: any) => s.kind === 14).map((s: any) => s.label); expect(keywordLabels).toContain('let'); expect(keywordLabels).toContain('true'); expect(keywordLabels).toContain('false'); expect(keywordLabels).toContain('nil'); }); it('should provide functions in suggestions', async () => { const provider = getExprCompletionProvider(); const mockModel: any = { getWordUntilPosition: () => ({ startColumn: 1, endColumn: 1, word: '' }), getValue: () => '', getLineContent: () => '', }; const mockPosition: any = { lineNumber: 1, column: 1 }; const emptyContext: any = {}; const emptyToken: any = {}; const result = await provider.provideCompletionItems(mockModel, mockPosition, emptyContext, emptyToken); const suggestions = (result && result.suggestions) || []; const functionLabels = suggestions .filter(function (s) { return s.kind === 9; }) .map(function (s) { return s.label; }); expect(functionLabels).toContain('len'); expect(functionLabels).toContain('filter'); expect(functionLabels).toContain('map'); expect(functionLabels).toContain('now'); }); });