First of all, feel free to join in on our weekly meetings!  Mail us for the zoom links.

• general & talk: Friday 11:45am – 1pm
• vr meetings: Friday 11am-11:30am
• language meetings: Thursday 12pm-1pm

Second, read some of the following papers. 

• http://h2r1.cs.brown.edu/wp-content/uploads/2017/03/whitney17.pdf
• http://h2r1.cs.brown.edu/wp-content/uploads/whitney18.pdf
• http://h2r1.cs.brown.edu/wp-content/uploads/arumugam18.pdf
• https://arxiv.org/pdf/1905.12096.pdf 

What do you like about them? What would you be more interested in focusing? Perhaps you don’t like that goals can be defined in a single way. Perhaps you want to think more about how we can represent the world and add a vision module to it. Let us know exactly what you want to invest your time in!

Third, do the tutorials [disclaimer: you don’t have to do all of them if you want to focus on one you really like right away, but if you are unsure we would suggest trying one of each type~]:

• AR/VR:

• • https://learn.unity.com/project/roll-a-ball-tutorial
  • Getting started with Magic Leap and Unity: https://creator.magicleap.com/learn/guides/unity-setup-intro
• ROS: 
  • ROS Overview: http://wiki.ros.org/ROS/Introduction#:~:targetText=ROS%20is%20an%20open%2Dsource,between%20processes%2C%20and%20package%20management.
  • Create a ROS package: http://wiki.ros.org/ROS/Tutorials/CreatingPackage
  • Simple Publisher/Subscriber: http://wiki.ros.org/ROS/Tutorials/WritingPublisherSubscriber%28python%29
    • • http://gazebosim.org/tutorials?tut=ros_overview 

• simple_rl:
  • Link: https://github.com/david-abel/simple_rl 

• • Simple_rl is framework for reinforcement learning and other types of sequential decision making agents. This is where you can play around with simulation of little agents in a gridworld. You will need to understand (maybe implement) the Value Iteration algorithm. You should also look read chapter 17 of Russell Norvig (the AI textbook).
• pomdp_py:
  •  Documentation: https://h2r.github.io/pomdp-py/html/
  • Paper: https://arxiv.org/pdf/2004.10099.pdf
  • To get started:
    • Follow the installation instructions.
    • Can you implement the Tiger domain yourself?
      • What is the optimal policy in the Tiger domain, and why?
    • Can you implement another domain yourself, either from the list here, or a POMDP you created on your own?
      • What is the optimal policy in a small version of your domain, and why?
      • Try to solve your domain by:
        • POUCT or POMCP
        • SARSOP (check out  solvers.py. )
        • What is the difference between POUCT and POMCP? between POMCP and SARSOP?
      • Try to implement a basic value iteration algorithm for POMDP.
             * See solution here.

• PyTorch: PyTorch is a deep learning framework like Tensorflow. Think of it as a package used in Python to create neural networks. Let us know if you need help with installing anything (venv, etc.)

• • ZeroToAll: (Deniz’s favourite) After you watch these videos and you go through the tutorials implement the I-DRAGGN model in this paper [http://h2r1.cs.brown.edu/wp-content/uploads/karamcheti17.pdf]
    • Github: https://github.com/hunkim/PyTorchZeroToAll
    • Youtube: https://www.youtube.com/watch?v=SKq-pmkekTk&list=PLlMkM4tgfjnJ3I-dbhO9JTw7gNty6o_2m
  • Seq2Seq: You could also learn how to implement a seq2seq and then implement the model in this paper [https://arxiv.org/pdf/1905.12096.pdf]
    • https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html 
    • https://github.com/bentrevett/pytorch-seq2seq/blob/master/1%20-%20Sequence%20to%20Sequence%20Learning%20with%20Neural%20Networks.ipynb 

• PiDrone: PiDrone is an autonomous raspberry pi/Python drone developed by the lab. Each year (traditionally, Fall semester) Stefanie runs a class (CS1951R) where every student builds their own drone and learns about the underlying autonomy. You can read papers about the drone here.:

• • http://h2r1.cs.brown.edu/wp-content/uploads/pidrone18.pdf
  • https://arxiv.org/pdf/1910.03516.pdf
  • Software stack (see README.md): https://github.com/h2r/pidrone_pkg

If you need any help with them we encourage you to drop by the lab (knock on the door) and introduce yourself, and work from the lab. Most people who are around are friendly, willing, and able to help! You can also reach out to the mailing list h2r at cs.brown.edu or:

• eric_rosen@brown.edu AR/VR – ROS – simple_rl related tutorials
• deniz_bayazit@brown.edu PyTorch – simple_rl related tutorials
• matthew_berg@brown.edu PiDrone – ROS – simple_rl tutorials

Did these feel like interesting problems to solve? Did you like the coding side of it? Do you feel like you could make use of these frameworks in your own dream project?

 

If you do these and you enjoy doing it, we can think about the next step and help you join the lab officially. If you decide you would like to join in on a certain project or perhaps start your own research group please talk to Stefanie (our PI!) who can be reached at stefie10@cs.brown.edu. If you can’t reach out to her you can also contact the members mentioned above. Our goal is to get you paired up with a student who is already in the lab, and attending one of our weekly project-specific meetings.  You will start out by helping with their project, and then move on to your own project. 

~ Extra Resources ~:

• Deep Learning (more mathy): https://www.deeplearningbook.org you can also get a Kindle on Amazon but this link is free!
• Understanding LSTM Networks (recurrent networks) https://colah.github.io/posts/2015-08-Understanding-LSTMs/ 
• Word Embeddings Explained: 1, 2, 3, 4.
• Seq2seq attention visualization (PyTorch diagram is not as clear): http://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/