XetHub, a Seattle-based company founded by Yucheng Low, Ajit Banerjee, and Rajat Arya—former Apple engineers responsible for scaling the company’s internal machine learning infrastructure—has been acquired by Hugging Face. XetHub specializes in enabling software engineering best practices for AI development, particularly by enhancing Git’s ability to scale to terabyte-sized repositories. Their expertise will be crucial in helping teams collaborate on large datasets and models.
XetHub, now part of Hugging Face, will play a pivotal role in the next phase of growth for Hugging Face’s datasets and models. By moving away from Git LFS and adopting XetHub’s more efficient storage backend, the platform will become significantly more capable of handling massive files often used in AI. For instance, instead of re-uploading an entire 10GB Parquet file when a single row is added, only the affected chunks will need to be re-uploaded, saving time and resources.
Looking ahead, this collaboration will also address challenges associated with working on large datasets and models, providing better solutions for team collaboration and data evolution tracking. Hugging Face is set to scale new heights with XetHub’s technology, particularly as the field progresses towards models with trillions of parameters.
