
We are releasing Decider v1.3 and Decider Flash v1.3, the successors to our earlier decision models: v1, v1.1, v1.2, and Flash v1.2. Decider Flash v1.3 is a smaller model distilled from Decider v1.3. Both models are available on Hugging Face under Apache license, through OpenRouter, and the Perplexity Decisions API. Perplexity Decider is inspired by Jev, TypeSafe AI’s System One Model.
It supports binary and multi-class classification with ordered or unordered labels, returning probabilities that applications can use to classify, route, rank, and evaluate inputs.Both v1.3 models improve performance and robustness across a broad range of text and vision tasks through bidirectional attention, better-curated training data, and a refined training recipe. We also introduce locate, a new feature to highlight the selected content for named entity recognition, personal information identification, and extractive question-answering tasks.In this post, we explain how we train and evaluate Decider and share how we use it internally at Perplexity.What is a decision model?A decision model is a general-purpose classifier. You give it an input and a set of questions each with its possible answers. For each question, it returns a probability distribution over the answers.
It supports 3 question types:Yes / No: binary decisions.Multiple choice: custom multi-class labels.Score: ordered levels.Decision models support a wide range of classification and scoring tasks:Judging outputs: assessing faithfulness, quality, and safety.Routing: directing requests to smaller or larger models based on difficulty.Moderation and security: detecting harmful content, phishing, and prompt injection in tool outputs.Filtering and tagging: organizing large collections of content.Retrieval and reranking: scoring search results by relevance.Many of these tasks were previously handled by generative models like LLMs, even though they do not require free-form text generation. Decision models sit between LLMs and task-specific classifiers. They return probabilities for multiple questions about the same input in a single forward pass, avoiding token-by-token answer generation and sharing computation across questions. Unlike task-specific classifiers, they can handle new tasks through natural language prompts, a flexibility learned from training across diverse tasks in a shared question-answering format.
TrainingTurning a generative LLM into a decision modelWe build Decider by adapting strong generative LLM backbones. To explain the adaptation, we first describe how a request is represented. As illustrated below, a request consists of a shared context (the state) and a number of questions. The state will be reused by all questions.
For example, given a document and its summary, a single request can ask whether the summary faithfully represents the document and whether it is intended for an expert or a general audience. After the state, each question block contains a question and its possible answers, each assigned a predefined label token such as A, B, or C. A special <|marker|> token indicates where to read out the model’s prediction. At this position, a learned linear output layer (a decision head), maps the hidden state to scores for the question’s answer options, which are then converted to probabilities.We initialize the output layer from the base model, which normally scores next-token from the entire vocabulary, and only keep the rows corresponding to the predefined option tokens such as A, B, or C.
This pruned layer is then trained to turn the hidden state of <|marker|> into answer scores. This approach covers all classification questions because they can all be formulated as a multiple choice.For training, we take hundreds of public datasets spanning diverse tasks (sentiment, natural language inference, intent detection, entity matching, moderation, fact checking, ratings, etc.) and convert each example into the unified format described above: a state, one or more questions, their options and a target. The target can be the single correct option if we have gold labels, or a probability distribution over options.In v1.3, we introduce locate, which extracts text from the state instead of selecting from predefined options. A locate question describes what to find, using either a short label such as “location” or an instruction such as “Extract the city name where the job is located.”
The model returns the exact matching text from the state, along with its start and end character locations. If it identifies no matches, it returns an empty list. Because of shared state, a single request can combine locate, classification, and scoring questions about the same input. For example, it can ask whether a job requires senior-level experience and extract the required skills.Bidirectional attention: letting the model read both waysOur models start from strong pretrained generative backbones, whose causal attention lets each token attend only to itself and preceding tokens.
Our ablation studies showed that making the backbone’s full-attention layers bidirectional improves performance, so we adopt this design to let each token incorporate context from the entire input.Bidirectional attention introduces another design choice: how should the state and questions interact? We considered three attention patterns, with different implications for performance, answer stability, and cache reuse:Everything sees everything (v1.2): The state and all questions are processed together with full bidirectional attention. Computation over the state is shared, but an answer to one question can change depending on what else is asked.Isolated questions (v1.3): The state attends to all the questions, while each question attends only to the state and its own tokens. Questions can still influence one another indirectly through the shared state but results are more stable.Fully cacheable: The state only attends to itself, while each question attends to the state and its own tokens.
The state is question-independent and reusable across requests to take advantage of prompt caching. We don't use it by default, because letting the state see the questions matters a lot for several tasks, but a small fine-tuning can switch a model to it.We ultimately chose isolated questions for their strong performance and stability.Training recipeTraining proceeds in three stages for both model sizes. We first train the larger model, based on Qwen3.8-27B, using cross-entropy on large datasets, then distill it into the smaller Qwen3.5-4B model.The first stage is generalist training, where we fully fine-tune the model on large-scale corpora. In this stage, we only accept data that helps the model across the board and is not specific to any benchmark; we thus exclude any benchmark training splits.After this generalist model is trained, we run a second, more fine-grained stage using LoRA.
The data used in this stage is targeted toward specific domains and tasks. We add data in the second stage rather than the first one if we observe narrow gains or if we observe some regressions on other tasks. We use LoRA to avoid drifting too much from the generalist model and risking losing performance to fit those specific domains. The goal is to improve performance while retaining the backbone’s general knowledge.
We also use a weight when merging the LoRA back into the base model, to control the drift more precisely and favor held-out metrics above any in-domain gains.Finally, we add a vision-specific step. Similar to the second stage, the vision step is treated as another fine-grained specialization step, using a LoRA adapter that is merged back to the base model with a weight. Some vision capabilities can come at the price of text performance. Since we observe much more usage of the models on text, we prioritize text metrics over vision metrics.For both text stages, we use the average weights from three runs, each using a different random seed.
We found this improves the performance and robustness of the aggregated model.Once the final large model is obtained, we apply the same three-stage training procedure to the smaller model. Here, the large model acts as the teacher: its predicted output distribution replaces the gold labels, and the smaller model is trained to minimize the KL divergence between its predictions and those of the teacher. This distillation objective outperforms cross-entropy on the same training data. While cross-entropy performs better on benchmarks with dedicated training data, it generalizes less well to held-out data.
We hypothesize that distillation better transfers the teacher’s output distribution, including its confidence over non-gold alternatives, whereas cross-entropy encourages overfitting to the benchmark-specific training distribution. We therefore adopt the KL-based distillation objective.To mitigate train-test contamination, we remove any training samples that could overlap with any evaluation we use. This is important because some public training datasets contain large parts of some evaluation datasets. We also conduct extensive bias audits and removal to make sure our gains are reliable and not just driven by learned biases.Adding the ability to extract informationThe locate feature extracts information such as named entities from text, without changing how it answers ordinary requests.
To do so, we introduce an additional training stage: we add a new special token, <|locate|>, a span head, and LoRA adapter to Decider. We initialize <|locate|> ’s embedding from <|marker|> learned in the previous training stages. We train only this new embedding, the span head, and the LoRA adapters, while keeping the original backbone weights and decision head frozen. Each requested field gets its own <|locate|> token.The span head uses <|locate|> ’s representation together with the state-token representations to score where matching spans start and end, and which start-end pairs belong together.
It also uses the <|locate|> token’s representation to predict field-specific thresholds for these scores. Selected spans are returned as character positions in the original text.We train the span head with an adapted InfoNCE loss that pushes every gold span above the threshold, and the threshold above every other span. The loss has two terms: The first one pushes the scores of correct spans above the threshold and the wrong spans. The second term pushes the threshold above the scores of wrong spans.
Fields with no answer get only the second term, which teaches the model to return an empty list.We train on a mix of span extraction tasks, including named entity recognition, PII detection and extractive question answering. We vary the field descriptions from short labels to detailed instructions, so the model learns to follow the request rather than memorize a fixed label set.Locate teaches the model a structurally different task, and we want to add it without losing what the model already does well. Merging the locate adapters into the backbone causes some regression on decision requests. To limit this, we are running investigations such as more replay data from earlier stages and auxiliary objectives that keep the model’s outputs on decision requests close to those of the original model.
However, we find that merging is currently still not lossless, so we keep the adapters unmerged and activate them only for locate requests.EvaluationWe evaluate Decider v1.3 and Decider Flash v1.3 on external benchmarks from the Hugging Face Decision Index, Vals AI’s ClaimBench, and Artificial Analysis Decision Index. Across these evaluations, our models demonstrate strong performance. Even though these leaderboards include privately held tasks, they can still be gamed through repeated submission and iterative tuning based on leaderboard feedback. To complement these leaderboards, we evaluate the models on an internal suite combining private production tasks with held-out evaluations dra
Texto de perplexity.ai tal como lo publicó, leído de su página y mostrado aquí con el formato de El Autómata. El original, en perplexity.ai · Volver a la noticia de El Autómata