Serving technology enthusiasts for more than 25 years.
TechSpot is the place to go for tech advice and analysis.
What’s important: Everyone is coming up with innovative ways to avoid the huge costs associated with training and creating AI models. Researchers have developed an open competitor to DeepSeek, which is said to match the reasoning abilities of OpenAI o1. Stanford and University of Washington researchers have developed a technique for creating a new AI system dubbed “s1.” The code and data that were used to build the model are already available on GitHub. A paperpublished last Friday explains how the team achieved this result through clever technical tricks.
Instead of training a reasoning-model from scratch, a costly endeavor that costs millions, they “fine-tuned” used distillation to take an existing language model. They extracted reasoning capabilities from Google’s AI model – specifically Gemini 2.0 Flash Thinking Experimental. The base model was then trained to mimic the step-by-step process of solving problems on a small dataset.
. This process involves explicitly instructing the model to reason by using curated examples. Their dataset was made up of 1,000 carefully selected questions from Google’s model. TechCrunch reports that the training process took 30 minutes using 16 Nvidia GPUs
. These GPUs are expensive – they cost around $25,000 each – but renting them costs less than $50 in cloud computing credits.
Researchers also discovered a neat tip to boost s1’s capabilities even more. The model was instructed to “wait” prior to providing its final solution. This command gave it more time to check their reasoning and arrive at slightly better solutions.
This model is not without caveats. Since the team used Google’s model as its instructor, there is a question that s1’s skills, although impressive for its low cost, may not yet be able scale up to the best AI has on offer. Google could also protest. It could be watching to see what happens with OpenAI.
