Summarizing Sam Bowman's Research Notes

I think these two notes: Note A and Note B are very useful guidelines for young researcher and researchers who get confused on what to do next like me, so I summarized them possibly in a paraphrased way.

For a successful project, you must:

1. Identify an open problem, present a hypothesis about it, and survey the relevant literature;
2. Design and run an experiment to test that hypothesis;
3. Analyze the results to reveal what your experiment tell us about your hypothesis.

Literature review

• Do this early! The reason:
  • Make sure that what you’re doing hasn’t already been done before.
    • If it has, and the paper is easy to find, you won’t get full credit;
  • Learn about common methods, datasets, and libraries that will make you life easier
  • Buy yourself more time to think about the questions that haven’t been answered in the literature

How

1. Do a keyword search on Google Scholar
2. Download the papers that seem most relevant
3. Skim the abstracts, intro, & previous work sections
4. Identify papers look relevant, appear often, & have lots of citations on Google Scholar
5. Download those papers
6. Return to 3

Some guidelines

• You’ll have to use your judgment in deciding which papers to read, and which papers to trust
  • Papers can disagree with one another
  • You might just find too many papers
• Which paper will be most useful?
  • Newer ones that build on the paper you like most
  • Peer-reviewed paper
  • Published papers with negative results
• Finding negative results
  • Negative results are often reported as brief mentions in a paper that has mostly positive results
  • Look for gaps in the literature
    • GANs are hot outside NLP, why?

Organizing a project and doing good science

---