ThirdAI provides an easy-to-use Python package with an interface that closely resembles the standard open source training frameworks (like Keras). We also have a built in model converter that can take in a Keras model and then train it using our BOLT engine. The sample training scripts for training a BOLT network and training a Keras model using our BOLT engine are shown on this page.
Is BOLT free to download and install for everyone?
As of now, a limited set of users will be given access to our BOLT package. Please apply for trial access here
Does BOLT support all types of models?
At present, BOLT supports Fully Connected Neural Networks. Soon, we will be adding functionality to train giant embedding models (like Facebook’s DLRM and Amazon’s DSSM), Graph Convolutional Networks (GCNs), Convolutional Neural Networks (CNNs), and more.
Are there any hyperparameters that a user needs to choose?
We have a single tunable hyperparameter called load_factor that determines how many computations BOLT needs to perform. A lower load_factor trains the model faster, but potentially at a lower accuracy gain per batch.
Is there any trade-off?
BOLT uses ever-so-slightly higher memory than a naïve model. However, compared with TensorFlow’s high level of model replication (effectively prohibiting large batch sizes on a single GPU), BOLT’s memory usage is much less, thereby allowing for unusually large batch sizes on large models. BOLT is an approximation and hence may require different fine tuning.
Is it true that you can train a 200 million network on a laptop?
Yes! On an M1 laptop with a mere 8 GB RAM, BOLT can train a 200 million parameter model 20% faster than an A100 GPU! Read details here.
