Computer science student researches large language models’ programming capabilities Meet Naveed Sekender

Mathematics was Naveed Sekender’s first passion in science, technology, engineering and math (STEM). He was captivated by math and logic growing up and saw each math solution as being a piece to a larger puzzle. Today, he is a computer science student at the University of California, Davis. Early in his undergraduate education Naveed began learning Python, and with it discovered a love for programming.

“What began as simple scripts gradually evolved into complex projects, making me realize the potential of combining data, algorithms and computational power,” said Naveed. “As my skills expanded to include C++, Tensorflow and Sci-Kit Learn, my passion for Deep Learning and Geo-spatial analysis grew significantly.”

He joined the National Nuclear Security Administration Minority Serving Institutions Internship Program (NNSA-MSIIP) under the guidance of his mentor, Dr. Pei-Hung Lin, in the Center for Applied Scientific Computing (CASC) of Lawrence Livermore National Laboratory (LLNL). The NNSA-MSIIP provides paid opportunities for undergraduate and graduate students at Minority Serving Institutions pursuing degrees in critical science, engineering, technology, mathematics and other disciplines that support the current and future missions of the NNSA.

Now, Naveed is taking that early passion for math and programming and is using it to study large language models (LLMs) like Microsoft’s CodeBERT, Meta’s LLaMA and OpenAI’s GPT-4 model. Specifically, he is looking at how these language models can assist in programming language related tasks, called programming language processing (PLP).

One strength of LLMs is their ability to generate text in various languages as well as translating languages. For example, generating a block of HTML code when asked for help building a website, or translating Spanish into English and vice versa. However, LLMs may make mistakes and generate plausible but incorrect responses, called hallucinations, due to inadequate training. Naveed hopes to utilize multiple LLMs at the same time to account for some of these problems with LLM code generation.

LLMs can be applied to specialized tasks through a type of technique called fine tuning. Naveed exploits machine learning APIs, developed by the company Hugging Face, to fine-tune and optimize the LLM software for better code generation and analyzation. Ultimately, he wants to create a unique set of APIs that will make LLMs more useful for programming language translation.

“The design and implementation of an intuitive API will democratize access to advanced machine learning capabilities in PLP,” explained Naveed “Developers, irrespective of their expertise in machine learning, will be able to seamlessly integrate these state-of-the-art models into their applications, fostering innovation and improving overall productivity.”

Because programming is so important to much of our society, from the economy, cybersecurity and even climate change modeling, Naveed expects a streamlined coding process using LLMs will be of great use for a variety of industries.

One of Naveed’s favorite parts of his participation so far was visiting Washington, D.C., where he was introduced to the diverse NNSA-MSIIP team. He is excited to be collaborating with other interns, scientists of other backgrounds and peers who help him discover more real-world applications for his project. The new experience he is gaining in machine learning will put him at the forefront of technology innovation, says Naveed.

Naveed also “resoundingly” recommends the NNSA-MSIIP program to other upcoming scientists. “It offers invaluable experience in a stable industry and provides the unique opportunity to network with incredibly talented individuals. It's a fantastic launchpad for both learning and professional growth.”

After he completes his participation, Naveed has plans to create his own artificial intelligence startup company and has already been slowly setting it in motion. He will take what he has learned as a NNSA-MSIIP participant and implement it into his future business model.

The NNSA-MSIIP Program is funded by NNSA and administered through the U.S. Department of Energy’s (DOE) Oak Ridge Institute for Science and Education (ORISE). ORISE is managed for DOE by ORAU.