About
Table of contents
In-class reading reactions
Every class will include assigned readings, usually 2, with the exception of classes where we will be presenting or discussing project updates (marked in the schedule). For each class with reading, I expect you to read at least one of the papers and be ready to discuss it.
When you arrive in class each day, you will see (on the board or projector) a discussion question based on the readings for that day. You will have the first 10 minutes of class to respond to this question. These are the reading reactions. Reading reactions will also serve as the attendance sheet for the class (see below on attendance policies).
Reading reactions are due after the first 10 minutes of class. After the first ten minutes, you can still submit the reading reaction (such as if you are absent that day), but it will not count toward your attendance. To accommodate conflicting deadlines, emergencies, or anything else that might come up during the semester, you are allowed to skip three reading reactions, no questions asked. You also do not need to submit a reading reaction for the papers you are a discussion lead on. If you are absent for a particular day, you can submit the reading reaction after class. The Reading Reaction assignment on Canvas for that day will be updated after class to include a google form link to submit your response. If you are leading a discussion, you do not need to prepare a discussion response. See below for what to prepare if you are leading a discussion.
Reading reactions format
The reading reaction will be about 1 paragraph (3-5 sentences), focused on responding to the question based on the readings, connections to other readings, and critical/thought-provoking questions you had. Please do NOT provide a summary of the papers. The reaction should include at least one direct reference (e.g., a quote, finding, result, or discussion) from at least one of the papers for that week.
Reading reaction grading
Reading reactions will be graded on a 2 point scale and mostly focused on completion. As noted above, you may pass on 3 of the reading responses. Any additional missed responses will receive a 0.
2 points = the reading reaction answers the posed question and includes at least one direct reference to one of the papers in that class.
1 points = the reading reaction answers the question but does not include a direct reference to a paper
0 points = no submission
Leading a class discussion
Each week, a group of students will lead the class in a ~1 hour discussion of the assigned papers for that week. You are expected to sign up to be a discussion lead once during the semester. Your responsibilities for a discussion lead are:
- Read and understand the assigned papers that week along with any supporting material that you think would provide helpful background. Please also read the optional readings for that day, if any.
- Present a 10-12 minute talk on the papers. Summarize the papers’ key ideas, research questions, methodology, and findings.
- Lead a class activity based on the papers, followed by a discussion about the activity. To keep the discussion going you should prepare 8-10 discussion questions about the papers. These should be creative questions that do not have easy answers. The questions should be the result of your attempts to understand the material, its implications, and its connections with other work.
- Double-check with the instructor at least 24 hours before class regarding your proposed group activity and the contents of your slides. This does not need to include your responses to reading responses. We will be able to provide feedback if you send us the materials the day before class.
- Some days we will have in-class assignments (see more details below) meant to guide the class through ideas from the papers. If you sign up for a day where we have such an assignment, instead of preparing an activity in advance, you will instead complete the assignment in advance (should take no more than 30 minutes), share your completed assignment with the instructor at least 24 hours before class and be prepared to help classmates with the assignment during class.
Below are ideas for group activities and general questions from past classes. Here are some group activities that worked well in prior classes:
- Role-playing activities: have students within groups analyze the same problem/question by assuming different roles as stakeholders: e.g., writers, news reporters, artists, patients or caregivers, moderators, developers, researchers, policy makers.
- Design activities: have students come up with new design ideas for additions/modifications to a tool or solution. How do we redesign existing applications, or design completely new ones, to implement ideas or address certain key challenges identified in the paper?
- Extending current research: have students collectively brainstorm how they would extend or modify the paper, and why. In addition, consider discussing the strengths and weaknesses of the current paper’s approach, and their favorite parts in the paper.
- Class surveys based on the paper (e.g., google forms): This has served as a great way to understand overall patterns and variations in class opinions regarding design and challenges in intelligent communication technologies.
Tips
Break students up into small groups (3-4 people max) Initiate discussion with example answers to the questions you pose. Introduce hypothetical or real-world examples (e.g. products or recent events) to frame the discussion Tie your questions to research that you or other folk may be conducting. Keep a timer.
Contribution statement
We expect both discussion leads to participate equally in the presentation, activity, and project. To help ensure equal participation. Each discussion lead should include 1-2 sentences documenting their specific contribution to preparing and leading the class discussion.
In-class assignments
There will be 2-3 assignments in class meant to actualize the findings or ideas from the readings. Completeing these assignments will be part of your particiption grade. These will predominalty be structured coding assignments where you are provided with some initial code that you will fill in to achieve some result. These assignments will be shared during class. We do not expect you to have any preparation for the assignments, and they will mostly be graded on completeness and effort over correctness. If you would like the assignments in advance please let the instructor know. If you are sick or absent the day of a class assignment, you can complete the assignment up to two weeks after the class and send it to the course staff.
Class attendance
I will take attendance for class each day based on reading reaction submissions. During presentation days, attendance will be recorded using a sign-in sheet. You are expected to attend and actively participate in every class session. To make the most of this class and not disrupt others, I expect you to be on time and ready to discuss the readings. If you can’t make it because of sickness or a family emergency or similar, please let me know before class.
Class Project
A major goal of the course is to help students critically reflect and build upon the ideas introduced in class. You will work on a group research project related to language, interfaces and communication throughout the semester. We have various milestones for the project below.
Project pitches
Pick and develop a research idea or question that interests you, develop your own views, and come up with pitches for two new research projects. The projects can fall into three broad categories:
A data project analyzing language use in some communication context (e.g., conversations on reddit, the wording of security and privacy instructions). Note that the communication does not have to be two-way, social or synchronous: it could be conversations on reddit (two-way, social communication), or newspaper articles (one-way).
A design project proposing a new system (or modifying an existing one) to enhance communication through language.
An evaluation project conducting some assessment on a system. The system could be interactive or generative, or both. Evaluations should include some human and some automatic components, depending on the type of system you are evaluating.
Use the research pitch session to get early feedback from the instructor and your peers on topic selection and potential resources for literature survey. Make sure to include research questions (RQs), why the problem is important (why), and how you plan to solve it in each research pitch (how). There are some great resources online on how to approach research and selecting what problems to work on (or not):
- Read this section on “Selecting Problems” on Page 6 in the “Syllabus for Eric’s PhD students”.
- First two sections of Daniel Fried’s lecture on conducting NLP research
- Two more sets of suggestions from other fields:
Time limits: Try to keep your pitch/presentation to 6-8 minutes in total. This will be followed by 3-4 minutes of Q&A from the instructors or your peers. Please practice beforehand to not run out of time!
Guidelines for project pitches
For each project pitch, submit 2-3 paragraphs that covers:
- Clear description of a new problem statement and the current gap that you aim to address through your research idea.
- An argument, with citations to relevant literature, that these are important research questions that we likely don’t have answers for yet.
- Initial thoughts on research questions and methods you would employ to answer these.
- Clearly articulate RQs/Why/How in pitches.
Project resources
We have compiled some databases, codebases, and compute credits that might be helpful for planning and executing your project. Here is a list of potential resources. Let us know if you have questions about any of it. Here are also some example projects from previous years.
Project proposal talk
After selecting one research pitch, come up with project proposal. Incorporate feedback from the class obtained during the pitches, and remind us how you propose to address any challenges during your proposal.
The proposal will include a written document (the project proposal report, covered below) and talk. For the talk, be sure to cover:
- Research questions that can be answered with empirical results (RQ/Why/How).
- A complete research study or system design. For example, coming up with either a quantitative study to answer clearly-defined research questions or building a new system, chrome plugin, or ML framework to solve some current challenge. You’re free to choose your favorite methods that are appropriate to answering your research questions as long as you justify your methods.
- For a data project: a plan for how you’re going to collect data and drawbacks to your approach.
- For a design project: the feasibility of your system design and how you would address any challenges that may arise.
- For an evaluation project: a plan for how you will collect initial data to analyze, such as how to recruit people or what metrics you will calculate.
- Timeline for the proposed project, clearly identifying next steps and possible roadblocks or risks.
Project proposal report
We will be using the ACL paper format for both the project proposal and final report. Your project proposal report will be a first draft of your final paper that you will continue to fill in over the course of the semester. The project proposal report should reflect what you presented in the proposal presentation. It should include an introduction, related work, proposed study design and methodologies, timeline, distribution of work among students, and expected project outcomes and contributions. For the project proposal, try to keep the PDF to under 7 pages.
Note: A single-paragraph collaboration plan will be required in each student’s project proposal to receive full credit. Eventually, a single-paragraph contribution statement summarizing each group member’s contribution to the project will be required in each student’s final paper.
What do you need to provide in the collaboration plan?
Each group member should include a single paragraph (ideally at the end of their proposal) documenting their group’s plan to complete the project collaboratively and specifically highlight their specific role as a collaborator. In other words, what is/will be your contribution to the group’s successful completion of the 1, project and 2, proposal document + final paper.
Think of this as an informal contract among group members.
Final Report
Every team will hand-in a report summarizing their findings. When writing up your findings, look at the research papers we have read in class. My expectation for the final project is that it significantly extends previous findings in an area within CS. Your research project should be a plausible attempt at contributing a novel result. The final write-up should have the quality of a publishable paper, with the contribution being roughly that of a short paper (e.g., a workshop paper). The papers should be a maximum of 10 pages. Aim for 7 to 10 pages.
For guidance on writing research papers, please refer to papers you enjoyed reading in class, here are also two links for writing HCI and NLP research papers.
Final Talk
You will also present your project in a final talk. The talk should be a total of 10-12 minutes. It should include:
- Problem statement. Your presentation should open with a short (i.e., < one minute) description of the problem you are trying to solve. This should be easy if you pick a pitch you’ve already made. You simply need to remind us of your problem. If you choose something you haven’t pitched in the past, then you can spend up to two minutes on the problem statement.
- Implementation (e.g., RQs, data, methods). Tell us about your overarching research design, research questions, data collection, how you answered these questions.
- Key takeaways and findings. You are free to reuse any of your presentation materials from earlier in the class.
Grading criteria
- Identifying an interesting problem.
- Quality of problem statement.
- Addressing problem statement with social data or a proposed system.
- Quality of presenting solutions in report.
- Thoughtful use of course readings in presentation.
- Satisfactory participation as a contributing member.
What do you need to provide in the contribution statement?
Each group member should include a single paragraph (ideally at the end of their final paper) documenting their specific contribution to the group’s successful completion of the: 1, project, and 2, proposal document + final paper. Include details from each stage of the project: ideation, formulation, execution, presentation.
I will be asking all group members to individually provide member evaluations to me along with individual contribution statements in the final papers, and allocating points based on satisfactory participation in group projects.