AI Singapore brings inclusive generative AI models to Southeast Asia

According to Amazon CTO Werner Vogels’ technology predictions for 2024, generative AI will become more culturally aware in the near future.

This involves training large language models (LLMs), a type of AI algorithm, on a diverse range of data, leading to more nuanced and accurate results. The goal is to make generative AI more accessible and useful to users worldwide.

However, LLMs rely on data from the internet, which is mostly associated with high-resource languages. This term typically refers to languages, such as English, that have an abundance of linguistic resources available for natural language processing tasks.

It is critical that organisations gain the ability to easily customise their LLMs with local data in their native languages to foster social inclusion, stimulate economic growth by opening up new markets, and create improved citizen experiences.

Culture influences everything. Recognising this, AI Singapore (AISG), a national programme launched by Singapore’s National Research Foundation to enhance the country’s AI capabilities, is making its LLMs more culturally accurate, localised, and tailored to Southeast Asia.

Collaborating with AWS, AISG developed SEA-LION, a family of LLMs that is specifically pre-trained and instruct-tuned (a powerful fine-tuning method that allows for greater control over LLM behaviour) for Southeast Asian languages and cultures.

SEA-LION also serves as the foundation for Singapore’s National Multimodal LLM Programme, contributing to the island’s capabilities in AI research and innovation. This initiative aligns with the National AI Strategy 2.0 that outlines plans to deepen the use of AI in Singapore.

The model will focus on more commonly used languages in Southeast Asia, including Bahasa Indonesia, Bahasa Melayu, Thai, and Vietnamese, and will eventually be extended to include other Southeast Asian languages like Burmese and Lao.

Building regionally represented LLMs also requires rich, hyper-local data in relevant languages. An example of local language nuances is the term “LOL” (an abbreviation of “laughing out loud” in English).

In Thailand, people commonly use “55555”, while Indonesians often use “wkwkwk.” LLMs trained on culturally diverse training data, like SEA-LION, enhances the ability of generative AI applications to grasp nuanced aspects of human experiences and navigate complex societal challenges.

SEA-LION will be available on Amazon SageMaker JumpStart this month. The platform provides pre-trained, publicly available models to help customers around the world get started with machine learning.

The relatively small 3-billion and 7-billion parameter variants of SEA-LION that have been released so far were trained using Amazon EC2, a service that provides scalable compute capacity in the cloud.

These smaller variants are designed to offer increased flexibility and accessibility compared to many commonly used LLMs available in the market today, which often boast hundreds of billions of parameters.

AISG will soon launch a commercial version of instruction tuning parameters for SEA-LION. This version aims to enhance the capacity to capture nuances in Southeast Asian languages, improve contextual understanding, enhance multilingual reasoning, and generate context-rich outputs.

Building, training, and deploying an LLM requires time, significant compute resources, and expertise. AISG is working with AWS to leverage the cloud’s infrastructure for tasks like ML training and high-performance computing using NVIDIA A100 Tensor Core GPUs, which deliver top-notch throughput and fast, responsive networking.

The compact size of SEA-LION also makes it relatively more cost-effective and efficient than larger LLMs, which can have hundreds of times more parameters.

Smaller LLMs allow developers to deploy faster, are cheaper to fine-tune, and perform more quickly during inference. The ease of deploying smaller LLMs on mobile devices, or at the edge, also helps businesses more easily adopt and create applications.

“Building an LLM requires reliable cloud infrastructure that is readily available exactly when needed, and AWS is instrumental in helping us scale cost-effectively. We built a 3 billion parameter LLM in just three months with AWS, and we have since scaled the model to 7 billion parameters, extending its reach to more audiences,” said Dr Leslie Teo, senior director of AI Products at AISG.

“By working with AWS, we can focus solely on training our models instead of managing infrastructure. This accelerates the development of unique LLMs that reflect our region’s diversity.”

As the digital transformation is accelerated, businesses in Vietnam are looking to improve their security capabilities, with AWS services being among the most optimal choices.

We hear from Andrew Sklar, director of Training and Certification at Amazon Web Services for Asia Pacific and Japan, about the tech giant's new training initiatives that are helping businesses and individuals in the region prepare for the next big evolutionary step in technology.

AWS re:Invent 2023 made a big impression with some groundbreaking announcements, thought-provoking keynote speeches, and immersive experiences to help businesses plan the future of their cloud transformation journey.