South Korea’s hottest AI voice assistant, GiGA Genie, converses with 8 million individuals every day.

The AI-powered speaker from telecom firm KT can management TVs, provide real-time site visitors updates and full a slew of different home-assistance duties primarily based on voice instructions. It has mastered its conversational abilities within the extremely complicated Korean language due to massive language fashions (LLMs) — machine studying algorithms that may acknowledge, perceive, predict and generate human languages primarily based on large textual content datasets.

The firm’s fashions are constructed utilizing the NVIDIA DGX SuperPOD information heart infrastructure platform and the NeMo Megatron framework for coaching and deploying LLMs with billions of parameters.

The Korean language, often known as Hangul, reliably exhibits up in lists of the world’s most difficult languages. It consists of 4 sorts of compound verbs, and phrases are sometimes composed of two or extra roots.

KT — South Korea’s main cellular operator with over 22 million subscribers — improved the sensible speaker’s understanding of such phrases by creating LLMs with round 40 billion parameters. And by means of integration with Amazon Alexa, GiGA Genie can converse with customers in English, too.

“With transformer-based models, we’ve achieved significant quality improvements for the GiGA Genie smart speaker, as well as our customer services platform AI Contact Center, or AICC,” stated Hwijung Ryu, LLM improvement group lead at KT.

AICC is an all-in-one, cloud-based platform that gives AI voice brokers and different buyer service-related purposes.

It can obtain calls and supply requested info — or shortly join clients to human brokers for solutions to extra detailed inquiries. AICC with out human intervention manages greater than 100,000 calls every day throughout Korea, based on Ryu.

“LLMs enable GiGA Genie to gain better language understanding and generate more human-like sentences, and AICC to reduce consultation times by 15 seconds as it summarizes and classifies inquiry types more quickly,” he added.

Training Large Language Models

Developing LLMs might be an costly, time-consuming course of that requires deep technical experience and full-stack expertise investments.

The NVIDIA AI platform simplified and sped up this course of for KT.

“We trained our LLM models more effectively with NVIDIA DGX SuperPOD’s powerful performance — as well as NeMo Megatron’s optimized algorithms and 3D parallelism techniques,” Ryu stated. “NeMo Megatron is continuously adopting new features, which is the biggest advantage we think it offers in improving our model accuracy.”

3D parallelism — a distributed coaching methodology wherein a particularly large-scale deep studying mannequin is partitioned throughout a number of units — was essential for coaching KT’s LLMs. NeMo Megatron enabled the group to simply accomplish this job with the very best throughput, based on Ryu.

“We considered using other platforms, but it was difficult to find an alternative that provides full-stack environments — from the hardware level to the inference level,” he added. “NVIDIA also provides exceptional expertise from product, engineering teams and more, so we easily solved several technical issues.”

Using hyperparameter optimization instruments in NeMo Megatron, KT skilled its LLMs 2x quicker than with different frameworks, Ryu stated. These instruments enable customers to robotically discover the very best configurations for LLM coaching and inference, easing and dashing the event and deployment course of.

KT can be planning to make use of the NVIDIA Triton Inference Server to supply an optimized real-time inference service, in addition to NVIDIA Base Command Manager to simply monitor and handle a whole bunch of nodes in its AI cluster.

“Thanks to LLMs, KT can release competitive products faster than ever,” Ryu stated. “We also believe that our technology can drive innovation from other companies, as it can be used to improve their value and create innovative products.”

KT plans to launch greater than 20 pure language understanding and pure language technology APIs for builders in November. The utility programming interfaces can be utilized for duties together with doc summarization and classification, emotion recognition, and filtering of probably inappropriate content material.

Learn extra about breakthrough applied sciences for the period of AI and the metaverse at NVIDIA GTC, operating on-line by means of Thursday, Sept. 22. 

Watch NVIDIA founder and CEO Jensen Huang’s keynote handle in replay beneath: