Data Science

  • 16 Posts
  • 9 Comments
Joined 2 years ago
cake
Cake day: June 17th, 2023

help-circle





  • From the article:

    DeepSeek-R1 release leaves open several questions about:

    • Data collection: How were the reasoning-specific datasets curated?
    • Model training: No training code was released by DeepSeek, so it is unknown which hyperparameters work best and how they differ across different model families and scales.
    • Scaling laws: What are the compute and data trade-offs in training reasoning models?

    These questions prompted us to launch the Open-R1 project, an initiative to systematically reconstruct DeepSeek-R1’s data and training pipeline, validate its claims, and push the boundaries of open reasoning models. By building Open-R1, we aim to provide transparency on how reinforcement learning can enhance reasoning, share reproducible insights with the open-source community, and create a foundation for future models to leverage these techniques.

    In this blog post we take a look at key ingredients behind DeepSeek-R1, which parts we plan to replicate, and how to contribute to the Open-R1 project













  • Creative people consistently say that they don’t spend a lot of time thinking about what they want to create. They just work on something. Often something nonsensical and useless. Sometimes something that’s meant to practice something they want to improve upon. Sometimes it’s half of an idea. Almost always it’s something that won’t ever be finished. In the process of working on whatever it is they’re engaged in, they get ideas for the next thing they want to work on. That’s how ideas come. Not from thinking about what the next idea will be, but by being engaged with an existing idea.

    An easy way to start is to start journaling. Write down something good that happed during your day. Elaborate on it. Write your thoughts. Don’t edit them or care about spelling or grammar. Just engage with your existing thoughts.