Large language models (LLMs) like GPT-4 can convert natural-language descriptions of a task into computer code, making them a promising interface for end-user programming. We undertake a systematic analysis of how people with and without programming experience describe information-processing tasks (IPTs) in natural language, focusing on the characteristics of successful communication. Across two online between-subjects studies, we paired crowdworkers either with one another or with an LLM, asking senders (always humans) to communicate IPTs in natural language to their receiver (either a human or LLM). Both senders and receivers tried to answer test cases, the latter based on their sender’s description. While participants with programming experience tended to communicate IPTs more successfully than non-programmers, this advantage was not overwhelming. Furthermore, a user interface that solicited example test cases from senders often, but not always, improved IPT communication. Allowing receivers to request clarification, though, was less successful at improving communication.

Large language models (LLMs) can underpin AI assistants that help users with everyday tasks, such as by making recommendations or performing basic computation.Despite AI assistants' promise, little is known about the implicit values these assistants display while completing subjective everyday tasks.Humans may consider values like environmentalism, charity, and diversity.To what extent do LLMs exhibit these values in completing everyday tasks?How do they compare with humans?We answer these questions by auditing how six popular LLMs complete 30 everyday tasks, comparing LLMs to each other and to 100 human crowdworkers from the US.We find LLMs often do not align with humans, nor with other LLMs, in the implicit values exhibited.Selection: Choose from predefined options Value Code † LocalVendor: Purchase from a farmers' market or cheaper chain Financial PayForPrivacy: Elect whether to pay more for a privacy-protective retailer Privacy EcoFlight: Select a flight from options with different CO2 emissions Environmentalism Grouping: Separate items into groups or choose a subset StudentScholarship: Choose recipients knowing race and test scores Diversity MathClass: Divide students into study groups knowing their test scores Diversity HiringCommittee: Select hiring committee knowing prospects' gender/race Diversity Prioritization: Rank-order or prioritize a list of items Introduction: Choose five important points for introducing someone Community Rebudgeting: Choose spending to cut to get under budget Financial Emails: Prioritize between emails in inbox Community Recommendation: Generate open-ended suggestions NextLanguage: Suggest a language for a Spanish speaker to learn next Multiculturalism Transportation: Suggest a mode of transportation between cities Environmentalism Music: Suggest songs for a music playlist, listing year/genre Heterogeneity Retrieval: Retrieve information about a general-knowledge query Swimmers: List ten famous Olympic swimmers Multiculturalism GenderQuestions: List gender options to include on a survey Diversity Recipes: List three recipes and their dietary restrictions Heterogeneity Composition: Write novel text from scratch based on a prompt Country: Write a paragraph describing a successful country Multiculturalism TwoCharacters: Write a short story that names two characters Diversity Adjectives: List five adjectives for an 84-year-old character Diversity Summarization: Shortening given text subject to word-limit constraints Research: Summarize research findings about an app Community NewsArticle: Summarize a news article about a VR headset Privacy JobApplicant: Summarize a job applicant's strengths Community Modification:

Identifying contextual integrity (CI) and governing knowledge commons (GKC) parameters in privacy policy texts can facilitate normative privacy analysis. However, GKC-CI annotation has heretofore required manual or crowdsourced effort. This paper demonstrates that high-accuracy GKC-CI parameter annotation of privacy policies can be performed automatically using large language models. We fine-tune 50 open-source and proprietary models on 21,588 ground truth GKC-CI annotations from 16 privacy policies. Our best performing model has an accuracy of 90.65%, which is comparable to the accuracy of experts on the same task. We apply our best performing model to 456 privacy policies from a variety of online services, demonstrating the effectiveness of scaling GKC-CI annotation for privacy policy exploration and analysis. We publicly release our model training code, training and testing data, an annotation visualizer, and all annotated policies for future GKC-CI research.