About

⚠️   Problem: Dialect differences cause performance issues for many types of users of language technologies. We show significant performance disparities in dialect QA, MT, and Semantic Parsing tasks. If we want fair and equitable NLP, system performance should be constant over dialect shifts to avoid allocative harms, which we call dialect invariance

📊   Data: Multi-VALUE is a suite of resources for evaluating and achieving English dialect invariance. It contains tools for systematically modifying written text in accordance with 189 attested linguistic patterns from 50 varieties of English. Researchers can use this to build dialect stress tests and train more robust models for their task using data augmentation.

Multi-VALUE: A Framework for Cross-Dialectal English NLP

🌐   Modeling: Multi-VALUE powers several plug-and-play methods to mitigate dialect discrepancies for different data scenarios. When aiming to enable a specific community of speakers, task-agnostic dialect adapters provide invariance using a small number of pairwise translations¹. However, language use varies across time, place, and social scenario creating many low-data scenarios. We work towards addressing these by integrating expert linguistic knowledge into Language Models with dynamic aggregation of linguistic rules² and model adaptation using a HyperNetwork condition on eWAVE features³.

Task-Agnostic Dialect Adapters for English¹

Dialect Adaptation via Dynamic Aggregation of Linguistic Rules²

Attribution: Multi-VALUE was designed and built at the SALT Lab 🧂 at Stanford University 🌲 and the Georgia Institute of Technology 🐝. This resource draws heavily from the Electronic World Atlas of Varieties of English (eWAVE), which aggregates the work of over 80 field linguists.

What Multi-VALUE can do for you

In short, Multi-VALUE can produce synthetic forms of dialectal text with modular control over which linguistic features are expressed. This will allow you to systematically isolate and the effects of certain syntactic and morphological structures on the performance of English NLP systems. Research applications include:

Getting Started

Use the following demo to start using Multi-VALUE.

Installation

		pip install value-nlp
	      

Demo

[VECTOR DIALECT] Hong Kong English (abbr: HKE)
      Region: South and Southeast Asia
      Latitude: 22.26
      Longitude: 114.25

'I talk with them yesterday.'

{(2,7): {'value': 'talk', 'type': 'bare_past_tense'}}

Important Considerations

Limitations

Additional Considerations

Please keep the following points in mind when using Multi-VALUE

Data Agreement

I will not use Multi-VALUE for malicious purposes including not limited to: deception, impersonation, mockery, discrimination, hate speech, targeted harassment and cultural appropriation. In my use of this resource, I will respect the dignity and privacy of all people.

References

	    @inproceedings{ziems-etal-2022-value,
	            title = "{VALUE}: {U}nderstanding {D}ialect {D}isparity in {NLU}",
	            author = "Ziems, Caleb  and
	            Chen, Jiaao  and
	            Harris, Camille  and
	            Anderson, Jessica  and
	            Yang, Diyi",
	            booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics",
	            month = may,
	            year = "2022",
	      }
	  

	    @inproceedings{ziems-etal-2022-multi-value,
	            title = "{M}ulti-{VALUE}: {E}valuating {C}ross-dialectal {NLP}",
	            author = "Ziems, Caleb  and
	            Held, William  and
	            Yang, Jingfeng  and
	            Dhamala, Jwala  and
	            Gupta, Rahul  and
	            Yang, Diyi",
	            booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics",
	            month = jul,
	            year = "2023",
	      }
	  

	    @inproceedings{held-etal-2023-tada,
	            title = "{TADA} : {T}ask {A}gnostic {D}ialect {A}dapters for {E}nglish",
	            author = "Held, William  and
	            Ziems, Caleb  and
	            Yang, Diyi",
	            booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",
	            month = jul,
	            year = "2023",
	      }
	  

	    @inproceedings{liu-etal-2023-dada,
	            title = "{DADA}: {D}ialect {A}daptation via {D}ynamic {A}ggregation of {L}inguistic {R}ules",
	            author = "Liu, Yanchen  and
	            Held, William  and
	            Yang, Diyi",
	            booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
	            month = dec,
	            year = "2023",
	      }
	  

	    @inproceedings{held-etal-2023-tada,
	            title = "Task-Agnostic Low-Rank Adapters for Unseen English Dialects",
	            author = "Xiao, Zedian and
	            Held, William  and
	            Liu, Yanchen  and
	            Yang, Diyi",
	            booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
	            month = dec,
	            year = "2023",
	      }