Discover and read the best of Twitter Threads about #ACL2023NLP

Most recents (2)

Excited to share our #NLProc #ACL2023NLP paper:

Controlled Text Generation with Hidden Representation Transformations

Work done during my applied scientist internship at @AmazonScience @alexa99 team.

Paper: arxiv.org/abs/2305.19230
Code: github.com/amazon-science… (Coming soon) Image
@AmazonScience @alexa99 LLMs are notoriously difficult to control. This work is an effort to fix that.

We create CHRT : a novel framework to attribute control LLMs using learned transformation blocks.

It can be used to minimize toxicity, maximize positive sentiment and more. Image
The approach has minimal loss in linguistic quality while achieving high attribute control.
Also has the least latency delta as compared to all other included baselines, something that makes it ideal for production environments. Image
Read 8 tweets
#ACL2023NLP
Do causal language models (CLMs) yield good representations with good isotropy and discrimination?

The answer is not always! To address the issue, our ACL2023 paper (arxiv.org/pdf/2210.01185…) proposes ContraCLM.

Joint work with @DejiaoZhang @nihalj_
We show that CodeGen (350M to 16B) pretrained on source code, and text-based CLMs (smaller than GPT2-Large) generated representations suffer from anisotropy and poor discrimination. Image
We show that ContraCLM enhances both isotropy and discrimination, regardless of whether the original CLMs suffer from the degenerated representations. Image
Read 4 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!