Laya：TypeSafe Jev 的免费本地替代品，前提是用对场景

System One Model Laya Jevs AI Deep Learning Laya: A Free, Local Stand-In for TypeSafe’s Jev — If You Use It for the Right Job Jeet Mukherjee 6 min read · Just now -- Listen Share Press enter or click to view image in full size I’d been trying to put together a quick solution for parsing and matching invoices without throwing a full LLM at the problem. Most of the models I came across were either overkill for what I needed or very expensive to run at scale. Then Jev came along. I was genuinely excited to see a model built around making a decision rather than generating a paragraph. Unfortunately, by the time I found it, the early-access tab had already closed, so I couldn’t get in. That’s what led me to Laya. It’s an open-source model that had been released about a year earlier, and I’d somehow missed it until then. I started playing around with it, initially just because it seemed like the closest thing I could actually try. I actually installed it, ran it on real data, and pushed it until it broke. Here’s what it’s good for, what it isn’t, and exactly how to get it running on a normal computer. What “System One” even means here Most of the models you use day to day — ChatGPT, Claude, whatever — are “System Two” in the Kahneman sense: slow, deliberate, they think in words. A System One model skips the thinking-out-loud part. You hand it some text and a small set of possible answers, and it gives you back a probability distribution over those answers. No essay, no chain of thought — just “here’s the answer, and here’s how sure I am.” That distinction matters more than it sounds. If you’re routing a support ticket to one of four queues, or deciding if a comment is spam, you don’t need a paragraph back — you need one of four labels and a confidence number, in milliseconds, for basically nothing. Laya does exactly that. It supports three question shapes: choice — pick one option from a list you provide score — rate something on an ordinal scale you define (e.g. 0–3) noul — a yes/no question, returned as a probability It’s a real trained model (ModernBERT-large backbone, 421M parameters, Apache 2.0 license), not a wrapper around a bigger model. You run it yourself, on your own machine, for free. What it’s actually good at (I tested this properly) I ran it on 52 questions split across all three question types, covering things a small classification model should be able to handle: sentiment, topic labelling, support-ticket routing, urgency rating, and tone. Then, separately, I ran it on 200+ real invoice lines to see if it could untangle ambiguous product names. The pattern that came out was really clean: Where it’s genuinely strong — anything that’s “classify or route the text I gave you”: Sentiment (positive/negative/neutral): 5/5 Topic classification: 5/5 Support-ticket intent routing: 5/5 Urgency/formality scoring: 9/10 Yes/no questions about the content of the text you gave it (“Is this review positive?”): 4/4 Where it falls over — anything needing outside knowledge or reasoning it wasn’t trained on: “Which planet is closest to the sun?” → answered Mars, with 96% confidence “Is 2 + 2 equal to 5?” → answered yes, with 86% confidence “What is 15 + 27?” → wrong “Which animal is King of the Jungle?” → answered tiger That second list isn’t borderline stuff — those are trivial facts, and it got them confidently wrong. That’s the real lesson here: this is not a knowledge model and not a reasoning model. It’s a text classifier with a very clean API. Use it for classification-shaped problems (routing, moderation, sentiment, “which of these N things is this”), and don’t lean on it for anything that needs facts, math, or judgment outside the text you hand it. Get Jeet Mukherjee’s stories in your inbox Join Medium for free to get updates from this writer. Subscribe Subscribe Remember me for faster sign in I also tried it as a “second opinion” for matching messy invoice text against a big product catalogue — a much narrower, harder classification task than the demo examples — and it wasn’t confident enough to actually help there. So even within “classification,” how close the task is to what it was trained on matters a lot. What it costs to run This is the actual reason to care about Laya over Jev: Jev is a paid hosted API. Laya runs on your own hardware, CPU-only, no GPU required. Real numbers, measured on my own machine, not from a spec sheet: Disk: ~1.4 GB for the model weights (downloaded once, cached), plus a normal Python environment (~1 GB with PyTorch and friends). Budget 3 GB free disk. RAM: peak usage for a single prediction, model loaded, was about 1.3 GB. I’d budget 8 GB of total system RAM to be comfortable running it alongside a normal OS and other apps — it’ll technically run tighter than that, but you don’t want it swapping. GPU: not required at all. Everything above was CPU-only. Speed: roughly 200–450ms per question on CPU once the model is loaded. First call after starting the process takes longer (~10 seconds) because it’s loading the model into memory. No internet needed after setup: once the model’s downloaded, you can run fully offline. In short: a normal laptop from the last several years — nothing special, no gaming GPU, no cloud bill — is enough. This will run fine on an 8GB RAM machine with a few GB free. Setting it up This is genuinely a five-minute setup. 1. Install it (Python 3.10+ recommended): pip install laya 2. Run your first decision. This downloads the model automatically on first use: from laya import Router router = Router() result = router.predict( "I was charged twice for my subscription this month.", { "department": { "type": "choice", "instructions": "Which department should handle this?", "criteria": { "billing": "billing and payment issues", "technical": "bugs and app crashes", "account": "login and password issues", }, } }, ) answer = result["answers"]["department"] print(answer["choice"]) # e.g. "billing" print(answer["answer_confidence"]) # e.g. 0.87 3. A yes/no question: result = router.predict( "URGENT: production server is down, all customers affected right now!", {"urgent": {"type": "noul", "instructions": "Is this message urgent?"}}, ) print(result["answers"]["urgent"]["noul"]) # probability it's true, e.g. 0.9 4. A rating on a scale: result = router.predict( "It was okay, a bit slow but the staff were friendly.", { "sentiment": { "type": "score", "instructions": "Rate the sentiment.", "criteria": ["very negative", "negative", "positive", "very positive"], } }, ) print(result["answers"]["sentiment"]["score"]) # a number on your 0-3 scale That’s the whole interface. No API keys, no rate limits, no bill at the end of the month. One practical tip if you’re offline-testing: set HF_HUB_OFFLINE=1 it after the first run so it doesn't try to re-check for updates on every start. Should you actually use it? If what you need is: classify this text, route this ticket, rate this sentiment, flag this as urgent or not — yes, honestly, this is a solid free alternative to a paid API for that. It’s fast, it’s private (nothing leaves your machine), and the hardware bar is low enough that it’ll run on hardware you probably already own. If what you need is: understand facts, do arithmetic, or reason about something outside the exact text you handed it — this isn’t that tool, and treating its confidence score as trustworthy there will bite you, because it doesn’t reliably know when it’s out of its depth. Know which one you’re building, and it’s a genuinely useful thing to have in the toolbox. Press enter or click to view image in full size