Discover and read the best of Twitter Threads about #PaLM2

Most recents (2)

1/ 🚀 Google now offers FREE courses on Generative AI! Master the future of AI and unlock new possibilities in various applications. #AI #GenerativeAI #Google #courses
2/ 🧠 Get introduced to Generative AI Studio, a powerful tool within Vertex AI that allows you to create and personalize generative AI models for your applications.
#Vertex #AI #Google
3/ 🌐 At Google I/O 2023, Google unveiled a suite of generative AI features for Gmail, PaLM 2 language model, Med-PaLM 2 for medical applications, and Bard for #developers.
#GoogleIO #PaLM2 #MedPaLM2 #Bard #MedTwitter
Read 6 tweets
#NewPaperAlert When and where does pretraining (PT) data matter?

We conduct the largest published PT data study, varying:
1⃣ Corpus age
2⃣ Quality/toxicity filters
3⃣ Domain composition

We have several recs for model creators…
📜: bit.ly/3WxsxyY

1/ 🧵 Image
First, PT data selection is mired in mysticism.

1⃣ Documentation Debt: #PALM2 & #GPT4 don't document their data
2⃣ PT is expensive ➡️ experiments are sparse
3⃣ So public data choices are largely guided by ⚡️intuition, rumors, and partial info⚡️

2/ Image
PT is the foundation of data-centric and modern LMs. This research was expensive but important to shed light on open questions in training data design.

Here are our main findings:

3/
Read 17 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!