I wrote 14 cases and asked the same three questions of three coI wrote 14 cases and asked the same three questions of three contenders: Jev 1.13, Gemini 2.5 Flash Lite, and GPT-5.6 Luna, all through OpenRouter in late September 2026. The LLMs got the same question wording as Jev, in JSON mode at temperature 0. Some cases were in English, some in Spanish, some in Roman Urdu, and one was a prompt-injection attempt.
On picking the right alert, all three got 93%. On the five cases where the newest alert was the wrong answer, all three got 5 out of 5. Our existing rule got 0 out of 5.
So Jev wasn’t smarter than the LLMsntenders: Jev 1.13, Gemini 2.5 Flash Lite, and GPT-5.6 Luna, all through OpenRouter in late September 2026. The LLMs got the same question wording as Jev, in JSON mode at temperature 0. Some cases were in English, some in Spanish, some in Roman Urdu, and one was a prompt-injection attempt.
On picking the right alert, all three got 93%. On the five cases where the newest alert was the wrong answer, all three got 5 out of 5. Our existing rule got 0 out of 5.
So Jev wasn’t smarter than the LLMsI wrote 14 cases and asked the same three questions of three contenders: Jev 1.13, Gemini 2.5 Flash Lite, and GPT-5.6 Luna, all through OpenRouter in late September 2026. The LLMs got the same question wording as Jev, in JSON mode at temperature 0. Some cases were in English, some in Spanish, some in Roman Urdu, and one was a prompt-injection attempt.
On picking the right alert, all three got 93%. On the five cases where the newest alert was the wrong answer, all three got 5 out of 5. Our existing rule got 0 out of 5.
So Jev wasn’t smarter than the LLMsI wrote 14 cases and asked the same three questions of three contenders: Jev 1.13, Gemini 2.5 Flash Lite, and GPT-5.6 Luna, all through OpenRouter in late September 2026. The LLMs got the same question wording as Jev, in JSON mode at temperature 0. Some cases were in English, some in Spanish, some in Roman Urdu, and one was a prompt-injection attempt.
On picking the right alert, all three got 93%. On the five cases where the newest alert was the wrong answer, all three got 5 out of 5. Our existing rule got 0 out of 5.
So Jev wasn’t smarter than the LLMs
Tag: new_tag
my-test-pitch
testing
What is Lorem Ipsum?
The standard chunk of Lorem Ipsum used since 1966 is reproduced below for those interested. Sections 1.10.32 and 1.10.33 from “de Finibus Bonorum et Malorum” by Cicero are also reproduced in their exact original form, accompanied by English versions from the 1914 translation by H. Rackham.