NVIDIA demonstrated four new inference platforms with DGX Cloud, including NVIDIA L4, which the firm claims offers “120x more AI-powered video performance than CPUs, combined with 99% better energy efficiency.” L4 can also be used to create AI videos, stream videos, encode and decode video, and more. Additionally, there is the 2D and 3D picture generation NVIDIA L40 as well as the LLM solution NVIDIA H100 NVL, which has 94GB of memory and an accelerated Transformer Engine. (That helps deliver 12-times faster GPT3 inference performance compared to the A100, according to NVIDIA.)
The last inference platform is NVIDIA Grace Hopper for Recommendation Models, which performs exactly what it says on the tin. Additionally, it can power vector databases and graph neural networks in addition to being designed for recommendations.
