Foundation Models, Instruction Tuning & RLHF

Large Language Models

Research into training, scaling, and deploying large language models that form the foundation of every Mentneo product.

Overview

Large Language Models (LLMs) are the core of modern AI systems. At Mentneo, we research the full spectrum — from pretraining on massive corpora to instruction fine-tuning, preference optimization (RLHF/DPO), and efficient inference.

Our LLM research spans architecture innovation (transformer variants, state space models), training stability at scale, and emergent capabilities. We publish both foundational and applied research to advance the field.

Key Research Topics

  • Transformer architectures
  • Pretraining at scale
  • Instruction fine-tuning
  • RLHF & DPO
  • Context length extension
  • Efficient inference (quantization, speculative decoding)
  • Emergent capabilities
  • Model evaluation & benchmarking

Products Using This Research

Mentneo LLM (flagship foundation model)Mentneo Chat (conversational AI)Code generationDocument understandingEnterprise automation

Large Language Models — Common Questions

A Large Language Model is a deep learning model trained on massive amounts of text data to understand and generate human language. LLMs learn statistical patterns across billions of tokens and can perform a wide variety of tasks — reasoning, coding, writing, summarization, question answering, and more — without task-specific training.

Mentneo trains LLMs using a multi-stage pipeline: (1) large-scale pretraining on diverse web and domain-specific corpora, (2) supervised fine-tuning on curated instruction datasets, (3) reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO) to align the model with human values and intended behaviors.

Mentneo LLM is optimized for enterprise and research use cases with a focus on factual accuracy, long-context understanding, multilingual capability, and efficient inference. We also provide fine-tuning APIs so enterprises can adapt the model to their specific domain data.

Related Research Areas

Reasoning ModelsAgentic AiRag SystemsEmbeddings