Getting Started with LLama 3: The Most Advanced Open-Source Model
Meta AI setting new standards in AI innovation
Getting started with the Llama 3 language model by Meta sounds exciting! Meta introduced the Llama 3 series last week, which is a significant advancement in language models featuring configurations of 8 billion and 70 billion parameters. The social media giant plans to release an even larger model of 400 billion parameter model in the future.
This release represents one of the most substantial developments of the year, with Meta simultaneously launching a series of open-source AI models, products, and research initiatives. This coordinated rollout underscores Meta’s commitment to leading the field in technological innovation.
Future enhancements for Llama 3?
- Meta plans to release models with over 400 billion parameters
- new capabilities as longer context windows
- additional model sizes
- Enhanced performance
- Comprehensive research paper.
Model Training and Data:
- Trained using 15 trillion tokens from public sources.
- Features a tokenizer with a 128K token vocabulary.
- Employs advanced filtering for high-quality data.
- Achieves over 400 TFLOPS per GPU on 16K GPUs.
Performance Benchmarks:
- MMLU: 8B model scores 68.4; 70B model 82.0.
- HumanEval: 8B achieves 62.2; 70B scores 81.7.
- GSM-8K: 8B at 79.6; 70B leads with 93.0.
- MATH dataset: 8B scores 30.0; 70B at 50.4.
Research Update
Meta will soon publish a paper detailing breakthroughs from the Llama 3 models. Notably, these models utilize a Tiktoken-based tokenizer with a 128K vocabulary and Grouped Query Attention, boosting efficiency and performance. Both 8B and 70B models continued improving beyond expected limits after training on 15 trillion tokens.
Access Details
- Fully open-source, including weights.
- Free access and integration.
- Supported on AWS and Google Cloud.
Significance
Meta’s commitment to open-source innovation is highlighted by the Llama 3 series. The forthcoming 400B model, already scoring 85 on MMLU, with upcoming features like multimodality and expanded context, promises to reshape the open-source arena.
Getting Started with Llama 3
Python and Huggingface
import transformers
import torch
model_id = "meta-llama/Meta-Llama-3-8B-Instruct"
pipeline = transformers.pipeline(
"text-generation",
model=model_id,
model_kwargs={"torch_dtype": torch.bfloat16},
device="cuda",
)
messages = [
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
{"role": "user", "content": "Who are you?"},
]
prompt = pipeline.tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
terminators = [
pipeline.tokenizer.eos_token_id,
pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = pipeline(
prompt,
max_new_tokens=256,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])
"Arrrr, me hearty! Me name be Captain Chat, the scurviest pirate chatbot to ever sail the Seven Seas! Me be here to swab the decks o' yer mind with me trusty responses, savvy? I be ready to hoist the Jolly Roger and set sail fer a swashbucklin' good time, matey! So, what be bringin' ye to these fair waters?"
You can play around with the Huggingface Demo
Meta AI Assistant, built with Llama 3
You can get started with Meta’s latest AI Assistant which incorporates LLama 3. Here are some of the features
On Facebook
Meta AI’s integration into Facebook enhances user interaction with their feed. If you stumble upon something intriguing, like a post about the northern lights, you can immediately use Meta AI to fetch additional details, such as the best times to witness the aurora borealis directly within your feed.
On WhatsApp
Meta AI on WhatsApp provides suggestions directly in your chats. For instance, you can ask Meta AI to recommend a scenic picnic spot, and it will promptly deliver suggestions, enhancing your planning experience right from your chat window.
On the Web
The new meta.ai website acts as your desktop assistant, ready to help with everything from math homework to drafting the perfect professional email. You can also save your chats with Meta AI, making it easy to pull up past advice whenever you need it.
Imagine Feature
The Imagine feature in Meta AI is about to transform how you create and visualize ideas. It’s designed to generate images from text in real-time, and it’s intuitive enough to adjust the visualization with each keystroke, providing a dynamic and interactive creative experience.
Also worth reading:
Meta Teaches AI To See The World Like Us With OpenEQA
Newsletter
One thought on “Getting Started with LLama 3: The Most Advanced Open-Source Model”