Category Archives: Writing

Checklist for Paper Submissions

Writing

We all want our papers to be awesome. Here is the checklist I use before paper submission.

  • Make sure there are no latex errors. Especially if you are using Overleaf, it will make a pdf even if your latex has errors, but it may not render the way you expect. If you are building on a PC, sometimes latex errors show up far up in the lag, so check for undefined references or citations or anything like that.
  • Spellcheck!
  • Read the bibliography and make sure there are no typos/errors/missing fields there.
  • Make sure the file size is 1MB or less. If it is larger, it is probably because of included images. Make sure 1) the resolution of the images is not too large (e.g., max 800 on a side), and 2) that if it is a photograph you are using jpg (which is compressed) and not png (which is losslessly compressed). However if it’s a diagram with lots of white space, use png, or even pdf or svg which will be imported into the generated pdf as a vector graphics image rather than pixles, and be small but look good at any resolution. Taking these steps will not change how it looks, but will dramatically reduce file size, making it take less time for you to upload, for reviewers to download, and for everyone else to download the paper for the rest of its lifetime.
  • Make sure all authors know they are authors and are okay with submitting.
  • Read the conference guidelines for authors and make sure you follow them all, especially as related to space. baseline stretch is now getting explicitly disallowed so be careful!
  • If double blind, make sure 1) you are citing all your own previous work, but you are citing it anonymously (as if someone else wrote the paper) 2) there are no other bits that deanonymize you.
  • Check the acknowledgements and make sure you acknowledge all funding sources (if not double blind).

Writing a Research Statement for Graduate School and Fellowships

Writing

Writing a research statement happens many times throughout a research career. Often for the first time it happens when applying to Ph.D. programs or applying to fellowships. Later, you will be writing postdoc and faculty applications. These documents are challenging to write because they seek to capture your entire research career in one document that may be read in 90 seconds or less.

Think of the research statement as a proposal. Whether you are applying for a Ph.D. program or for a faculty position, you are trying to convince the reader to invest in you. To decide whether to make this investment they need to know three things: 1) what will you do with the investment, 2) why is that an important problem? and 3) what evidence is there that you will be successful in achieving the goals that you have set out.

The first paragraph, therefore, should describe what problem you are aiming to solve, and why it is an important problem. One common failure mode is to be too general and vague in this paragraph. Yes we all want to solve AI! But you want to write about your specific take, angle, or approach. This will set up the rest of the statement, about why you are the one person uniquely qualified to solve the problem you set up here. Sometimes people shy away from being too specific, because they worry that it will put them in a box. Don’t worry! Research interests always evolve, and you will not be signing in blood to do this exact research plan. It is better to ere on the side of being too specific because it shows you can scope out an exciting project and that you have good ideas, even if you are not sure that this specific idea is the one you will eventually pursue.

The next paragraphs should describe your past work as it fits into the research vision you have outlined in the first paragraph. You can start with a paragraph for each project or paper you have worked on. The paragraphs can be more or less the abstract for the paper . However you should be clear exactly what your role in the project was, give credit to collaborators, and spend more time on the parts of the project you contributed to directly. You also need to tie it to the research vision in paragraph 1. The strongest statement presents your life, as an arrow that points unambiguously towards solving the research question you have outlined in paragraph 1. Of course, no one’s life is actually an unambiguous arrow! However I think it helps to think that way because you are trying to tie the projects together to show how they have prepared you and furthered you along the research trajectory. Even if this project wasn’t directly connected in terms of its research questions, you can write about how it taught you technical tools that you can apply to your research objective, or how it taught you something that led to your current research objective.

The last paragraphs should describe concretely what you plan to do next. If you are applying to a Ph.D. program, you should name the groups you wish to work with and explain why they are a good fit for you. For a fellowship, you should describe why this work is a good fit for the work done by the organization you are applying to.

Writing a Technical Paper

Musings, Writing

This post describes my outline/structure for a technical paper. I have found a fairly specific structure works best for abstracts and introductions. For the other sections, I describe a process for writing them.

Abstracts and Introductions

The first sentence should be a general statement of the problem’s importance.  The second sentence should summarize the shortcomings of previous work. It’s very tempting to make this go on for more than one sentence, but don’t give in – just one sentence! You will have more space later.  The next few sentences should describe your technical solution to the problem.  The last two or three sentences describe the evaluation.  Usually one sentence about the quantitative evaluation, and one sentence about the robot demonstration.  Did it work? How do we know that it worked? This does not need to describe the whole evaluation, but should cover the most impressive thing the robot can do as a result of the paper.
I think this abstract is a good example of following this template:

For introductions, the pattern is similar.  The first paragraph should generally describe the problem, focusing on the problem, what it is, and why it is important.  The second paragraph should succinctly describe related approaches and why they do not solve the problem.  The next few paragraphs should describe our technical approach for solving the problem.  The last paragraph should describe the evaluation and how we show that we have solved it.   I like to have a figure on the first page of the paper (Figure 1) that summarizes or shows what the robot can do that it couldn’t do before.  This figure is often the first thing reviewers see, and should show the “impressive result” of the paper visually, often a storyboard of the video.

Related Work

The aim of a related work section is to situate your contribution with respect to other people’s work.  You want to give the reader an overview of papers that are solving similar problems or using similar methods.  You also want to tell the reader why your contribution is new, different, and better.  

Before starting, always always always use natbib.  Even if the conference template claims not to support natbib, you can engineer it in anyway, and you should.  Natbib contains \citet which is a “noun phrase” citation such as “Tellex et al. (2019) wrote that”.  It will do “et al.” with no period after et, and a period after al. It will always spell the author’s name right if your bibtex entry is right.   Natbib saves time, makes your paper have fewer bugs, and avoids annoying people by spelling their name wrong (as long as you are careful to get it right once in the bibtex entry). 
I always start by writing two sentences about each paper.  The first sentence is a one sentence summary of the paper.  Example:  “\citet{tellex11} showed a framework that learns to map between language and robot trajectories functions, enabling a robotic forklift to understand natural language commands about movement and manipulation.”  The second sentence is a statement of the differences between their paper and your paper.  For example, “Our approach is different because it requires less annotated data and maps to reward functions instead of trajectories, enabling the robot to interpret human goals.”    I start by writing a long list of these sentences, and collect them in the related work section of the paper.  It’s okay if they are not organized.  Every time someone tells me about a paper I should put in related work, I add one of these sentence pairs (or at least a note to go look up that paper).  Your goal is to not miss any areas of related work that a reviewer might be familiar with.  
As you build up longer lists of citations and sentences in this format, you can start to group them into areas.  This might already happen naturally as you follow chains of related citations.  At some point, you can explicitly start grouping the sentences together into paragraphs.  It will still be choppy, but at least related papers will be together.  Once you have paragraphs, you can start to write summary sentences.  “There are many papers that learn to map between human language and robot trajectories~\citep{tellex11, matuszekxx, …}.  Our approach is different because it requires less annotated data than these methods and maps to reward functions instead.”    At this point you have a viable related work section. 


Depending on your available space, you may find yourself needing more summary and analysis sections that group papers together.  For example, you may not have space to give a one sentence overview of each paper.  Then you can cut the detailed versions, and just keep the high-level summary.  However, I have found that the best way to create the summary version is to start with the detailed version and then generalize.  If you do find yourself cutting the detailed versions, make sure to comment it out, or copy it into the draft journal paper submission for later. 
Sometimes, there will be a very closely related paper that needs a whole paragraph on its own.  For example, in our OO-POMCP paper, we built closely on the POMCP algorithm, and needed to give a detailed overview of the whole algorithm.  Or there might be a particularly close piece of related work with a subtle distinction, and you need to give more detail to express this distinction.  Feel free to write long if this is the case.  Sometimes you can figure out a way to cut later, and sometimes the detail is necessary. 


Related work is crucial for reviewers, who are often already familiar with the field, and don’t want to have to read your whole paper in order to understand your contribution.  Writing related work often helps tease out the novel aspects of the paper that should be emphasized in the introduction and abstract.  Don’t save it until the end.

I like to put related work after the introduction, before technical approach. I believe the introduction should explain enough about the paper that you can then review related work, and that it’s helpful to situate the paper with respect to related work next because that’s part of what I read to decide whether to read the whole paper. However it’s also valid to put it after technical approach, before the conclusion. Then you can assume the reader has a deeper technical understanding of the work and give a more nuanced presentation of its differences to related work.

Technical Approach

Think of the paper as a tree of problems and solutions.  It is important to not give any technical material with out first explaining the motivation for that material.   Technical/mathematical material is a solution to some problem, and it is extremely difficult to understand without first being clear about the problem it is solving.   First is the overarching problem the paper is solving, which you give in the abstract and introduction.  Then in technical approach you break the problem down into sub-problems.  Each of these sub-problems has a solution, which itself can be a set of sub-sub problems paired with technical solutions.  At any point, the reader may decide to stop descending delving into the leaf nodes and go on to the next sub-problem in the tree, and the paper should be designed for this kind of approach.  Actually only the very rare reader will descend into all the full technical detail of the paper – it is only when implementing the paper that someone typically does this.  
Even when implementing and going all the way to the leaf nodes, it is crucial to explain the problem first, then the solution. This approach situates the technical material into the larger ark of the paper, and helps people attach it to a larger structure in their brain about how the paper is put together.  

I like sentences like “In order to make the robot do X, we need Y and Z.” Then explain that bit. Then “In order to do Y we need A and B.” The whole paper should fit into a tree of these sentences, where the root of the tree is the overall motivation/main technical contribution for the paper.


A common failure mode, of what not to do, is to put “preliminary” or “background” material into an introductory section of the paper, that reads like a laundry list of random techniques.  When a background section makes sense, it should be situated as part of the tree of problems and solutions for the overall paper, but that this particular section is not novel to this paper.  It should still be connected to the rest of the paper as a tree of problems and solutions.   It should not be a laundry list of technical material that will come up later, because people won’t know what it’s for at this point in the paper, and won’t remember it. 

Results

For the results section, I like to start with a sentence describing the aim of the evaluation. This sentence should tie things back to the motivation set out in the introduction and the abstract. The results section should deliver on the promises made in the introduction. What did the robot actually do, that it couldn’t do before? How do we know that it did it? When reading a paper, often I will skip to the results section, especially if i am confused about the technical approach to find out what the robot actually did. This paper has a good example of this approach. For each subsection or set of experiments, open it with the sub-hypothesis that that part of the evaluation is showing. A common pattern is to have some results in simulation, and a demonstration on the real robot; explain how each piece of the evaluation demonstrates the robot’s new capabilities. Make sure that the metrics you are using are as easy to understand as possible. For example you could report “reward” but that requires the person understands and agrees with your reward function; it’s often better to report “accuracy” whether the robot did the right thing or not over many trials.

Avoid writing a description of one experiment, then a description of another experiment, then the results of the first experiment and then the results of the second experiment. After understanding what the first experiment is, give the results right away while it is still in the reader’s head. There is no reason to wait to give those results, when they will forget the details of the experiment before they get to the results. Moreover hopefully your separate experiments each tell a different part of the story about how the system works.

Conclusions

The first paragraph of the conclusion should summarize the contributions of the paper. This is different from the abstract in that you want to explain what you did in the paper: what results have been proved or what algorithms have been created, and what demonstrations have been done to show that it worked. The later paragraphs should cover future work. This part of the paper can recognize limitations of your paper and sketch out open problems that remain to be solved. It is okay if you do not plan to solve them; you are also laying out boundaries for what this paper does not solve, which helps people understand what it does solve. Moreover you are also describing possible follow-on work that may inspire someone to expand on this paper.