What pre-training, post-training and evaluation actually involve — and what they mean for compute investment, data governance and talent decisions in Ireland and the EU.
Large language models are now embedded in enterprise software, from customer service to code generation. Understanding how they are built is no longer a technical curiosity for engineering teams alone — it shapes decisions on vendor selection, infrastructure investment, data governance and risk. This briefing sets out the three stages of LLM training in plain terms, then examines what each stage means for organisations operating under EU data protection and AI regulation, and for the Irish market specifically.
Training a large language model happens in three distinct stages. Each has a different goal, a different cost profile and a different set of risks.
Pre-training teaches a model the mechanics of language. The model is given a sequence of text and trained to predict the next word, over and over, across a very large volume of publicly available text. Because the correct answer is always the next word already present in the source text, no manual labelling is required — this is what makes pre-training a self-supervised process, and what allows it to run at a scale no team could label by hand.
In the process of learning to predict text accurately, the model implicitly absorbs patterns in grammar, writing style and general world knowledge. The engineering challenge is significant: models are too large to fit on a single chip, so the workload is split across thousands of interconnected GPUs, with automated recovery to survive hardware failures and dedicated pipelines to process petabytes of source data.
The result is a model with broad language competence and world knowledge, but no reliable sense of how to behave as an assistant. Pre-trained models, on their own, can be capable but unpredictable — which is why almost no one interacts with one directly.
Post-training is where a raw pre-trained model becomes something usable: safe, well-formatted and responsive to instructions. It typically combines two techniques.
| Technique | What it does | Role |
|---|---|---|
| Supervised fine-tuning (SFT) | Trains the model on curated prompt-and-answer examples that demonstrate the desired behaviour. | Sets the floor — basic format and competence. |
| Reinforcement learning (RL) | The model generates candidate answers; a separate reward model scores them, and the model is updated to favour higher-scoring answers. | Raises the ceiling — beyond what curated examples show. |
Where SFT teaches a model to imitate good examples, RL lets it discover better answers than exist in any curated dataset. In practice, most current models rely on some combination of SFT followed by RL, though the exact balance differs between labs and remains an open question.
Every training run needs a way to tell whether it worked. Pre-training is measured directly: is the model getting better at predicting the next word? Post-training is harder to measure, because "helpful", "safe" and "well-formatted" are judgement calls, not scores.
For well-defined tasks such as maths or coding, models are tested against a set of problems with checkable answers. For everything else, teams first define what a good response looks like, then score model outputs against that definition — usually with a second LLM acting as an automated judge, spot-checked by humans. This produces a repeatable signal for whether a change to training data or method is actually making the model better.
The mechanics of training are the same everywhere. What differs is the regulatory, infrastructure and talent context an organisation is training or deploying within — and Ireland sits at an unusual intersection of all three.
None of this requires a CXO to train a model personally. It does mean three questions belong on the leadership agenda when evaluating any AI initiative.
Avanti Technologies advises on evaluating, deploying and governing AI systems for organisations operating in Ireland and the EU.
What machine learning actually is, where it earns money in a 20–250 person European business, what a first project costs, and what the EU rulebook asks of you.
Where deep-tech AI startups should own the stack, where they should rent it, and where ownership has quietly stopped being an option — with decision protocols.
How AI and software engineering cut avoidable cost and shorten lead times for Irish and EU metal fabricators — with a reference architecture and pilot plan.