Exploring the Innovations of Macrocosmos and TensorFlex: Advancements in Subnet 9 and Decentralized AI

The Rise of Macrocosmos: A Team with a Vision

Macrocosmos is a team formed by members of the original Subnet 1 team, including Stefan and Will, who were instrumental in its early development. Initially part of OpenTensor's Synapse team, they played a critical role in designing Subnet 1’s mechanism before it launched in October. As OpenTensor evolved, Macrocosmos spun off to bring a higher level of professionalism, organization, and scalability to the development of BitTensor and its subnets.

The growth of Macrocosmos is impressive, scaling from a small team to over 20 AI specialists and engineers in a short period. Their mission is to bring intelligence-driven solutions to BitTensor and compete with centralized AI systems on a global scale. This expansion reflects their confidence in the potential of BitTensor, especially as they look to build subnets that will power the future of decentralized AI.

Will, one of the co-founders, shared his background, transitioning from civil engineering to AI, with a particular focus on building subnets that make BitTensor a decentralized supercomputer. Stefan, his long-time friend and co-founder, added to the technical expertise with a PhD in subatomic physics and a passion for AI research, making the partnership a formidable force in AI innovation.

Subnet 9: Pushing the Limits of Decentralized AI

At the core of Macrocosmos’ current efforts is Subnet 9, a decentralized AI network that has shown incredible promise in recent competitions. Subnet 9 operates on a novel validation mechanism that uses the Refined Web dataset, developed by the UAE, to compare and evaluate AI models. Validators in Subnet 9 randomly sample data from this large dataset and compute the loss for each submitted model, rewarding the models with the best performance.

This competitive environment has seen remarkable success from the TensorFlex team, who have emerged as the frontrunners in decentralized AI competitions on Subnet 9. Not only have they outperformed other models like Falcon 7B, but they've also surpassed fine-tuned versions of LLaMA 2 in previous competitions. This demonstrates the power of decentralized AI, proving that it can compete with and outperform even state-of-the-art centralized models.

TensorFlex’s Journey on Subnet 9

TensorFlex’s accomplishments on Subnet 9 are nothing short of impressive. Their team, consisting of AI researchers and developers, has continuously refined their models to maintain their competitive edge. They manage various development operations while building a suite of products, including a liquid staking token on Ethereum and a validator, with plans to launch their own subnet soon.

Sing, the Director of AI at TensorFlex, highlighted the technical aspects of Subnet 9’s validation process. Validators pull trained models from platforms like Hugging Face and evaluate them against the Refined Web dataset. The dynamic nature of this competition ensures that only the best models survive, forcing participants like TensorFlex to innovate constantly.

One of the critical insights gained from their journey was the importance of model architecture design. Early on, deeper models proved more effective, but as TensorFlex scaled up to larger models with 7 billion parameters, they discovered that increasing the width of the embedding and MLP dimensions was key. This discovery, alongside fine-tuning hyperparameters like learning rates, allowed TensorFlex to stay ahead of the competition.

The Role of Incentives in Driving Innovation

A significant driver of innovation within Subnet 9 is the incentive mechanism that BitTensor provides. This system incentivizes participants to continuously improve their models to remain competitive. The competitive nature of the system has even led to TensorFlex holding off on submitting newer models, as doing so could risk losing their top spot. However, the threat of competitors training on their models and edging ahead by 1.5% on the leaderboard keeps the pressure on, ensuring that innovation never stops.

The collaborative genetic search algorithm at the heart of Subnet 9 allows for continuous improvement. The best-performing models are cloned and enhanced by validators, creating a system of dynamic, real-time innovation. This fitness function, coupled with a changing dataset and weight limits, ensures that models are always evolving, making it an elegant way to explore the vast and complex space of AI research.

Expanding the Horizons: Protein Folding and Beyond

Macrocosmos is not limited to Subnet 9; they are actively building other subnets that tackle different areas of AI research. One of the most exciting projects is Subnet 25, which focuses on protein folding. This subnet is designed to solve complex biological problems like protein folding more efficiently than centralized systems like AlphaFold 3. AlphaFold 3 is limited by its centralized servers and caps on protein size, whereas Subnet 25 can scale infinitely due to its decentralized nature.

By leveraging incentivized computing, Subnet 25 aims to solve protein-folding problems with a fraction of the resources used by centralized systems. This decentralized approach shows that BitTensor’s system can outperform even well-funded, state-backed AI efforts.

Looking Forward: The Future of Decentralized AI

As Macrocosmos continues to build subnets and push the boundaries of decentralized AI, there are several exciting developments on the horizon. Subnet 9 is set to receive a major upgrade with the introduction of the FineWeb dataset, which is five times larger than the current dataset. This upgrade will push the limits of what decentralized AI can achieve, enabling new models to outperform centralized systems even further.

In addition, the parameter cap for models on Subnet 9 will continue to increase, allowing for even more complex AI models to be trained. The decentralized and incentivized nature of BitTensor ensures that innovation will continue to drive AI research forward, with Subnet 9 leading the way.

Macrocosmos is also focused on making decentralized AI more accessible and visible to a wider audience. By improving dashboards and communication tools, they aim to highlight the value of Subnet 9 and showcase the exciting models that are being produced.

Conclusion

The work being done by Macrocosmos and TensorFlex on Subnet 9 is a testament to the power of decentralized AI. Through innovation, collaboration, and a competitive incentive structure, they are proving that decentralized systems can compete with and even outperform centralized AI models. As BitTensor continues to evolve, the advancements in Subnet 9, protein folding, and other subnets will shape the future of AI research, pushing the boundaries of what is possible in the decentralized AI landscape.

Source : @Opentensor Foundation