Announcement: PocketLLM was featured on ProductHunt! | Check out the latest AWS blog on ThirdAI benchmarks 

ThirdAI’s BOLT Deep Learning Engine Achieves Exceptional Performance on AMD EPYC™ Processors, Surpassing State-of-the-Art GPU Models

June 13, 2023 (Houston, TX)ThirdAI, a pioneering startup dedicated to democratizing machine learning capabilities, has successfully completed benchmarking of its BOLT deep learning engine on AMD EPYC™ 9004 Series processors as well as the latest 9004 Series AMD EPYC processors based on the “Zen 4c” architecture. These benchmarking experiments showcase BOLT’s exceptional performance acceleration on cutting-edge AMD EPYC CPUs versus current state-of-the-art models trained on NVIDIA GPUs.

In a series of rigorous tests, ThirdAI BOLT engine’s acceleration on AMD EPYC CPU-based machines outperformed well-established baselines across diverse machine learning tasks on NVIDIA A100s GPUs.

Graph node classification (a technique to detect fraud or scams within social media): using the open-source Yelp-Chi fraud detection dataset, BOLT surpassed accuracy measurements against well-established graph neural network baselines GCN (Graph Convolutional Networks) and GAT (Graph Attention Networks).

Trained on Yelp-Chi fraud dataset:

  • BOLT: 91.1% accuracy with training time of 10s on AMD EPYC 9754 CPU
  • GCN: 63.6% accuracy with training time of 150s
    • *training time 4.51s on NVIDIA A100 GPU
  • GAT: 81.4% accuracy with training time of 200s
    • *training time 22.5s on NVIDIA A100 GPU

Text Classification (technique for purposes such as sentiment analysis or intent prediction): evaluating ThirdAI BOLT against state-of-the-art pre-trained GPU-based model RoBERTa, BOLT achieved similar accuracy on two representative datasets, but at a fraction of the training time. This represents nearly a 200x speedup over GPUs.

Trained on Yelp Polarity and Amazon Polarity:

  • BOLT (AMD EPYC 9654 CPU): 92.3% accuracy (not pre-trained) with 130s total training time
  • RoBERTa (AMD EPYC 9654 CPU): 94.5% accuracy (pre-trained/tuned) with 9.1 hrs training time
    • *RoBERTa (NVIDIA A100 GPU): 1.77 hrs total training time
These benchmarking experiments underscore the remarkable performance and acceleration delivered by ThirdAI’s BOLT engine on the latest 4th Gen AMD EPYC processors. With the power of ThirdAI’s dynamic sparsity and AMD EPYC high performance cores and memory, we can deliver the fastest AI acceleration in the industry,” says Anshu Shrivastava, CEO of ThirdAI.
“The results reinforce ThirdAI’s commitment to democratizing machine learning capabilities and enabling cost-effective training and deployment of large language models.”