About

Reading remarks
1. Reading remark format
Leading a class discussion
Class participation
Class Project

Reading remarks

Every class will include assigned readings. For each class, I expect you to write a reading remark to one of the assigned readings, with the exception of classes where we will be presenting or discussing project updates (marked in the schedule).

Reading remarks are due by 7 pm the day before each class. This is to give the discussion leads time to read the responses and prepare discussion questions for class. There are a total of 16 reading responses, to accommodate conflicting deadlines, emergencies, or anything else that might come up during the semester, you are allowed to skip 3 reading remarks, no questions asked. You do not need to submit a reading remark for the papers you are a discussion lead.

Submit via Canvas discussion. If you are leading a discussion, you do not need to prepare a remark (see below for what to prepare if you are leading a discussion). The Canvas discussion will be closed after 7 pm the day before class. If your reading response is not in by then, you will recieve a 0 on that reading remark.

Reading remark format

For each class, you can either reflect on one of the papers for class that day, or respond to two other classmates’ reflection. Below we describe what each format looks like. You only need to do one format (either reflection or response) for each class. As a general rule, the reading remarks are there to help you remember the paper and to help the discussion leader summarize different views. When in doubt, please ask me! Send me an email or slack with your remark and I will take a look.

Reflection

Write 1-2 paragraphs (5-10 sentences), focused on what you learned from one of the readings, connections to other readings, or critical/thought-provoking questions you had. Please do NOT just provide a summary of the paper. Remember that everyone else will also be reading the same papers. To give you an idea about what a reflection could look like, here are some (non-exhaustive) options:

Option 1: One paragraph summarizing the contributions of the paper and one paragraph about potential improvements, such as how someone could build on the existing work.
Option 2: Two paragraphs that describe one or two themes you noticed in the paper. This is not a summary of the paper, but should instead answer questions like: what is the overall topic of the paper?
Option 3: Two to three thoughtful discussion questions based on the paper.
Option 4: Two paragraphs reflecting on this paper could aid you in your own research. Given enough of an introduction to your research so that other students will see how the paper fits in with your work.
Option 5: Two paragraphs introducing a new paper that you think builds on the class paper. Briefly summarize the new paper and explain what new ideas it introduces to the current reading.

Response

Please respond to two other classmates’ reading reflections (or responses). This response should be 1 paragraph (3-5 sentences). Your response can answer questions posed by the original reflection, ask follow-up questions, or reflect on a new paper introduced by the reflection. The response should NOT just say something like “interesting idea” you should include your own thoughts, questions, or ideas. You can also respond to another classmate’s response (e.g., as a threaded discussion).

Reading remark grading

Reading responses will be graded on 0-3 point scale. As noted above, you may pass on 3 of the reading responses. Any additional missed responses will receive a 0.

points = the reading remark satisfies one of the options above, is 5-10 sentences in length (total in the case of responses), well-written, and thought-provoking (i.e., you got some ideas from it).
points = the reading remark is too short/long, or does not show that you have engaged with the paper.
point = the reading remark is lacking two or more of the above requirements.
points = no submission

The options for reading remarks mentioned above will be equally graded.

Leading a class discussion

Each week, two to three students will lead the class in a ~1 hour discussion of the assigned papers for that week. You are expected to sign up to be a discussion lead once during the semester. Your responsibilities for a discussion lead are:

Read and understand the assigned papers that week along with any supporting material that you think would provide helpful background. Please also read the optional readings for that day, if any.
Summarize the papers’ key ideas, research questions, methodology, and findings. This should be a maximum of 10-12 minutes for both papers.
Lead a class activity based on the papers.
Lead a discussion of the readings based on the reflections that day.

Double-check with the instructor at least 24 hours before class regarding your proposed group activity and the contents of your slides. This does not need to include your responses to reading remarks.

Below are ideas for group activities and general questions from past classes. Here are some group activities that worked well in prior classes:

Role-playing activities: have students within groups analyze the same problem/question by assuming different roles as stakeholders: e.g., writers, news reporters, artists, patients or caregivers, moderators, developers, researchers, policy makers.
Design activities: have students come up with new design ideas for additions/modifications to a tool or solution. How do we redesign existing applications, or design completely new ones, to implement ideas or address certain key challenges identified in the paper?
Extending current research: have students collectively brainstorm how they would extend or modify the paper, and why. In addition, consider discussing the strengths and weaknesses of the current paper’s approach, and their favorite parts in the paper.
Class surveys based on the paper (e.g., google forms): This has served as a great way to understand overall patterns and variations in class opinions regarding design and challenges in intelligent communication technologies.
Games and Camp activities: Modifications of games like two truths and a lie, pictionary, and telephone. Can be surprisingly fun and informative. For example, telephone with machine translation, or figuring out how to prompt a model to give two truths and a lie about a paper.

Tips:

Keep a timer during the presentation and activity. Paper presentations should take no more than 12 minutes to give time to activities and discussion.
Break students up into small groups for class activities (3-4 people max)
Go around and check in with each group during activities. If a group is doing something interesting, ask them if they are ok being called on to share after the activity.
Introduce hypothetical or real-world examples (e.g. products or recent events) to frame the discussion
Tie your questions to research that you or other students may be conducting.
To keep the discussion going, read all reading remarks by the other students that week and integrate them into the discussion. You should prepare 8-10 discussion questions about the papers. These should be creative questions that do not have easy answers. The questions should be the result of your attempts to understand the material, its implications, and its connections with other work.
Be ready to call on people at the start of the discussion. If you don’t feel comfortable cold-calling on anyone, you can ask people beforehand if they are willing to be called on to share something from their reading reflection.

Contribution statement

We expect both discussion leads to participate equally in the presentation, activity, and project. To help ensure equal participation. Each discussion lead should include 1-2 sentences documenting their specific contribution to preparing and leading the class discussion.

Signing up for a discussion lead

Everyone must sign up for one discussion lead during the semester. You are required to lead a discussion with at least one other student. We will share the sign-in sheet in the first day of class. Anyone who has not signed up by the end of the first week of class will be assigned randomly to a discussion lead slot.

Class participation

Because this is a discussion-based class, you are expected to attend and actively participate in every class session. Attendance will be taken in class. To make the most of this class and not disrupt others, I expect you to be on time and ready to discuss the readings. If you can’t make it because of sickness, a family emergency, or similar, please let me know before class. If you have more than 5 unexcused absences during the semester, each additional missed class will incur a 5 point deduction of class participation (class participation is out of a total of 100 points, representing 15\% of your grade in the class).

Class Project

A major goal of the course is to help you critically reflect and build upon the ideas introduced in class. In a team of three to four students, pick and develop a research idea or question that interests you, plan a study or analysis, execute on your method, and report and discuss your results. The class project will have many sub-assignments and check-ins to ensure you are making progress on the project throughout the course. The project is meant to expose you to doing research in either HCI or NLP. To get some ideas, you can look at the last year or two of conference proceedings in either an HCI venue (like CHI) or an NLP venue (like ACL).

Finding a group for the class project

All class projects will be done in groups of three to four. You will have the first two weeks of class to find and join a group for the project. If by end of the two weeks you don’t have a group, we will assign you one.

Project pitches

Pick and develop two research ideas or questions that interests you, develop your own views, and come up with pitches for research projects. Use the research pitch session to get early feedback from the instructor and your peers on topic selection and potential resources for literature survey. Make sure to include research questions (RQs), why the problem is important (why), and how you plan to solve it in each research pitch (how). There are some great resources online on how to approach research and selecting what problems to work on (or not):

Read this section on “Selecting Problems” on Page 6 in the “Syllabus for Eric’s PhD students”.
First two sections of Daniel Fried’s lecture on conducting NLP research
Two more sets of suggestions from other fields:
- Four Mechanisms for Finding (and Being Found by) Research Problems
- Four golden lessons

We will share project pitches in two in-class pitch sessions that will include one-on-one discussion with other project groups and the intructor. Both the in-class pitches and written submission will be graded.

Guidelines for project pitch submission

For each project pitch, submit on Canvas 2-3 paragraphs that covers:

Clear description of a new problem statement and the current gap that you aim to address through your research idea (examples).
An argument, with citations to relevant literature, that these are important research questions that we likely don’t have answers for yet.
Initial thoughts on research questions and methods you would employ to answer these.
Clearly articulate RQs/Why/How in pitches.

Guidelines for in-class project pitch

You will be asked to submit feedback to other groups for the in-class project pitch sessions.

Project resources

We have compiled some databases, codebases, and compute credits that might be helpful for planning and executing your project. Here is a list of potential resources. Let us know if you have questions about any of it.

Project proposal

After selecting one research pitch, come up with proposal for a new research study or system design. Incorporate feedback from the class obtained during the pitches.

The proposal will include a written document (the project proposal report, covered below) and a talk. Details below.

Project proposal talk

The project proposal talk should introduce your research questions and specify how you plan to answer the questions. The talk should be 9-10 minutes, with 5 minutes of Q&A afterward. To prepare you for both the final project presentation and for presenting your own research at conferences, we will keep strict time and cut you off at the time limit. For the talk, be sure to cover:

Research questions that can be answered with empirical results (RQ/Why/How).
A complete research study or system design. For example, coming up with either a quantitative study to answer clearly-defined research questions or building a new system, chrome plugin, or ML framework to solve some current challenge. You’re free to choose your favorite methods that are appropriate to answering your research questions as long as you justify your methods.
For a data project: a plan for how you’re going to collect data and drawbacks to your approach.
For a design project: the feasibility of your system design and how you would address any challenges that may arise.
Remind us of feedback from your pitch, and how you propose to address any challenges.
Timeline for the proposed project, clearly identifying next steps and possible roadblocks or risks.

Project proposal report

You can use the CHI or ACL paper format for both the project proposal and final report. Be sure to use the two column format for both versions.

Your project proposal report will be a first draft of your final paper that you will continue to fill in over the course of the semester. The project proposal report should reflect what you presented in the proposal presentation. It should include an introduction, related work, proposed study design and methodologies, timeline, distribution of work among students, and expected project outcomes and contributions. For the project proposal should be 5-7 pages, not including references.

Note: A single-paragraph collaboration plan will be required in each student’s project proposal to receive full credit. Eventually, a single-paragraph contribution statement summarizing each group member’s contribution to the project will be required in each student’s final paper.

What do you need to provide in the collaboration plan?

Each group member should include a single paragraph (ideally at the end of their proposal) documenting their group’s plan to complete the project collaboratively and specifically highlight their specific role as a collaborator. In other words, what is/will be your contribution to the group’s successful completion of the 1, project and 2, proposal document + final paper.

Think of this as an informal contract among group members.

Final Report

Every team will hand-in a report summarizing their findings. When writing up your findings, look at the research papers we have read in class. My expectation for the final project is that it significantly extends previous findings in an area within CS. Your research project should be a plausible attempt at contributing a novel result. The final write-up should have the quality of a publishable paper, with the contribution being roughly that of a short paper (e.g., a workshop paper). The papers should be a maximum of 10 pages. Aim for 7 to 10 pages.

For guidance on writing research papers, please refer to papers you enjoyed reading in class, here are also two links for writing HCI and NLP research papers.

Final Talk

You will also present your project in a final talk. The talk should be a total of 10-12 minutes. It should include:

Problem statement. Your presentation should open with a short (i.e., < one minute) description of the problem you are trying to solve. You simply need to remind us of your problem. If you choose something you haven’t pitched in the past, then you can spend up to two minutes on the problem statement.
Implementation (e.g., RQs, data, methods). Tell us about your overarching research design, research questions, data collection, how you answered these questions.
Key takeaways and findings.
Connection to prior literature.

You are free to reuse any of your presentation materials from earlier in the class.

Grading criteria

Identifying an interesting problem.
Quality of problem statement.
Addressing problem statement with social data or a proposed system.
Quality of presenting solutions in report.
Thoughtful use of course readings in presentation.
Satisfactory participation as a contributing member.

What do you need to provide in the contribution statement?

Each group member should include a single paragraph (ideally at the end of their final paper) documenting their specific contribution to the group’s successful completion of the: 1, project, and 2, proposal document + final paper. Include details from each stage of the project: ideation, formulation, execution, presentation.