AWS Inferentia

Auto Added by WPeMatico

How Rufus doubled their inference speed and handled Prime Day traffic with AWS AI chips and parallel decoding

Amazon Elastic Container Service, AWS Batch, AWS Inferentia, AWS Trainium, Generative AI, Technical How-to /

Large language models (LLMs) have revolutionized the way we interact with technology, but their widespread adoption has been blocked by high inference latency, limited throughput, and high costs associated with text generation. These inefficiencies are particularly pronounced during high-demand events like Amazon Prime Day, where systems like Rufus—the Amazon AI-powered shopping assistant—must handle massive scale […]

How Rufus doubled their inference speed and handled Prime Day traffic with AWS AI chips and parallel decoding Read More »

Cost-effective AI image generation with PixArt-Σ inference on AWS Trainium and AWS Inferentia

AI/ML, AWS Inferentia, AWS Neuron, AWS Trainium, Generative AI, Neuron, Technical How-to /

PixArt-Sigma is a diffusion transformer model that is capable of image generation at 4k resolution. This model shows significant improvements over previous generation PixArt models like Pixart-Alpha and other diffusion models through dataset and architectural improvements. AWS Trainium and AWS Inferentia are purpose-built AI chips to accelerate machine learning (ML) workloads, making them ideal for

Cost-effective AI image generation with PixArt-Σ inference on AWS Trainium and AWS Inferentia Read More »

Optimizing Mixtral 8x7B on Amazon SageMaker with AWS Inferentia2

AI/ML, Amazon EC2, Amazon SageMaker, AWS Inferentia, AWS Neuron, Generative AI, Technical How-to /

Organizations are constantly seeking ways to harness the power of advanced large language models (LLMs) to enable a wide range of applications such as text generation, summarizationquestion answering, and many others. As these models grow more powerful and capable, deploying them in production environments while optimizing performance and cost-efficiency becomes more challenging. Amazon Web Services

Optimizing Mixtral 8x7B on Amazon SageMaker with AWS Inferentia2 Read More »