Task data and fine-tuning drove the performance gains
Piero Molino says the vast majority of value in his experiments came from data for the task and from fine-tuning after training. He describes the base model as more or less an independent variable.
Supporting evidence
Piero Molino on AI in Games
Original excerpt
The vast majority of the value was coming from the data for the task. The base model was more or less an independent variable. The actual effect was coming from the fine-tuning and the increased performance that you get after post-training.
Share insightCheck this claim