2022 in Review: The Year of ROS 2

Welcome to my 2022 recap. This year was quite busy — a little too busy, actually — and as such went by extremely quickly.

I had the privilege of being featured in two video series this past year. First was the Learn Robotics and AI podcast episode hosted by Chandandeep Singh, which was recorded at the end of 2021. This three-past series discusses my career progression and thoughts on robotics education. Later in the year, I was invited to present at an online seminar on Behavior Trees in Robotics, hosted by Professor Petter Ögren. This was a more technical talk which summarized my MIT CSAIL project from 2020, which you can also read about in this blog.

In last year’s recap post, I mentioned an interest in technical projects and exposure to ROS 2. I’m happy to say I got what I asked for, both from personal projects and my career. During the first half of the year, I scratched my open-source itch by launching an robot behavior simulation tool. Going into the second half of the year, I realized this was the kind of work I was truly passionate about. So, I ended up making the tough decision of leaving Boston Dynamics for a new role at PickNik Robotics, where I spent the remainder of the year getting acclimated. Finally, I capped off the year by trying out Advent of Code for the first time, learning a new programming language (Rust) in the process.

If you’re interested in anything you’ve read so far, keep reading to find out more!

---

Personal Open-Source Projects

pyrobosim

I remain proud of the work I did on home service robotics while I was at MIT CSAIL in 2020, and it was always in the back of my mind to take some of the robot-agnostic portions of the work and open-source it. Lo and behold, the first half of my year involved exactly this in the development and release of pyrobosim.

You can read the post from earlier this year which describes the entire process. To summarize here, this effort gave me a chance to revisit my past work in a less time-sensitive manner. Additionally, when one looks back at old code, good ideas on how to do things differently always surface. Besides the actual refactor, I got to:

• Solidify my understanding of Task and Motion Planning (TAMP) through the two other posts I did this year on task planning and TAMP, which used pyrobosim to show concrete examples that you can all run on your computers.
• Add completely new functionality, such as an actual interactive GUI, basic simulation backend, ROS 2 support, exporting worlds to the latest Gazebo version (which I’ve been told we’re not calling Ignition anymore), and more hooks into motion planning and execution.
• Stand up a GitHub repository the “right” way by setting up unit tests, continuous integration, documentation, and encouraging open-source contributions.

Since the release of pyrobosim, and the associated blog posts I linked above, there have been a few updates to share here:

• Two other people contributed to pyrobosim in 2022!
• Two students reached out to me over the course of the year. One was interested in TAMP and the other in multi-robot simulation. In sharing their ideas with me, they accelerated my existing plans to flesh out the TAMP and multirobot functionality in the 3 biggest post-release pull requests (#14, #17, and #28). Whenever these folks’ research comes to the public, I look forward to sharing their work with you all.
• One of my colleagues at PickNik, Dr. Andy Zelenak, started exploring pyrobosim … while I was interviewing there! When I first saw a documentation pull request from a PickNik engineer come into my repo, I was 99% convinced I was being grilled. However, I later found out that Andy has had a genuine interest TAMP, and we’re now collaborating on something that you will ideally hear about this year. Stay tuned!

TurtleBot3 Behavior Demos

TurtleBot3 behavior demos is a repository I started in 2021 to demonstrate ROS + Docker workflows and Behavior Trees — two very unrelated topics that happened to fall under the same code base. A year later, there is still some action here! Specifically,

• This became my first GitHub repo to cross 100 stars!
• Upgraded from ROS 1 Noetic to ROS 2 Galactic (#6), and shortly after moved to ROS 2 Humble (#12). In the move to ROS 2, I was able to grab some snazzy updates to the py_trees tools, as well as bring in Nav2 to double up on behavior tree usage.
• Switched from the archaic Docker + Make workflow that I learned in the past (sorry, Greg) to Docker Compose (#17). This is something I now need to do with other repos…

(Un)fortunately, Davide Faconti has been busy and since my upgrade a few months ago released new versions of BehaviorTree.CPP and the accompanying editor, Groot. So another update is in the cards at some point. I won’t say no to open-source contributions to help me out here… just putting it out there.

Thanks to everyone who looked at these examples and has kept me on my toes with issues, questions, and ideas. Really, this is testament that behavior trees are continuing to grow in popularity in robotics, as was analyzed in this 2020 paper by Razan Ghouli et al. As you read the next section, you’ll encounter another data point for this trend.

---

Moving It Over to PickNik

As I wrote at the start, this year came with another career change in my joining PickNik Robotics at the end of September 2022. You may know PickNik as the company that makes MoveIt, the de facto ROS package for motion planning. It was not easy to decide to leave the powerhouse that is Boston Dynamics, especially given all the great things I learned and talked about in last year’s recap, but ultimately I did so for a few reasons:

• PickNik’s philosophy aligned better with my desire to be more open with my work and share with the overall robotics community — something for which this blog has been an outlet. Granted, Boston Dynamics has since launched its AI Institute to also push on that front, which has been amazing to see and I look forward to seeing their contributions to basic research.
• If you’ve been reading my blog or following me on social media, you know that I’ve been keen about ROS, behavior trees, and overall robotic software systems architecture for a while now. For various reasons including the actual work, company size, and project lifecycle stage, I found that the opportunities to do more of what I wanted were more readily available elsewhere.

So… what have I been working on at PickNik these last 3 months?

pick_ik

My initial project involved helping rewrite an inverse kinematics (IK) solver that had already been put to practice. Specifically, this is the BioIK algorithm published by Philipp Ruppel at the University of Hamburg. In brief, BioIK is a “best of both worlds” IK solver that combines a gradient-based local optimizer and an evolutionary algorithm to break out of local minima. Critically, the solvers are made generic enough so that users can create custom, arbitrary cost functions to express goals beyond simple inverse kinematics. You can read the ICRA 2018 paper or Sebastian Starke’s thesis from 2020 for more details.

There currently exists a bio_ik package maintained by PickNik, which contains MoveIt IK plugins for ROS 1 and 2, but it was effectively a wrapper around the existing implementation. In parallel, one of my colleagues, Tyler Weaver, had been starting a clean rewrite of an algorithm based on this, and I was given the chance to see it to the finish line.

This project threw me head-first into a meaty C++ project with a clear goal of having a working IK solver. Tyler had already developed some of the basic functionality – the cost functions and the local optimizer – and being the experienced developer he is, stood up a nice functional interface that was delightful to work with. My tasks were then as follows:

• I first helped flesh out the unit tests to determine whether the local solver was actually working. Our tests used a simple 3-link planar robot model and the canonical Franka Emika Panda robot model used in the MoveIt tutorials, which is a 7-degree-of-freedom arm. In the process, I got to experience the Catch2 test framework for C++, which I enjoyed better than GoogleTest (aka GTest).
• Then, the evolutionary algorithm portion had to be written. Funny enough, my Master’s Thesis involved these for robot gait generation, but back in a time where they were still called genetic algorithms. Here, I got to poke through two different implementations in the code base, plus the literature, and reconcile all the inconsistencies into a working system. Thanks to Tyler’s initial work, the modular software interface and unit tests for the local solver served as a starting point to do the same with the added global optimization layer.
• Lastly was the addition of parallelism, which was the most significant “performance hack” applied by the original BioIK authors. This is the idea that you can run the evolutionary algorithm on independent, differently-seeded populations on multiple threads. To do this, I used a recently developed thread-safe queue implementation from PickNik’s ROS Support Library (RSL) package. This let me configure the solver to either immediately end all threads once the first solution was found vs. waiting for all threads to finish and then returning the best solution.

We recently released this package under the name pick_ik, announcing it on ROS Discourse just this past December. I hope you get to try it out and throw some issues and pull requests our way soon.

If I had infinite time and the chance to change one thing in the future, it would be to replace the local solver, which uses a simple (yet shockingly effective) numerical perturbation approach right now with a “real” nonlinear optimization tool such as NLopt. Other IK solvers out there such as TRAC-IK do exactly this, so there is precedent.

MoveIt Studio

Most of my time at PickNik so far has been spent working on the new MoveIt Studio product that was released this past year. Greatly simplifying things, MoveIt Studio provides

• A Web based front-end to remotely monitor and control a robot, from simple tasks like teleoperation to more complex autonomous behaviors.
• An interface to author behavior trees, made with a combination of pre-existing behaviors, or supplemented with custom behaviors.
• Access to the underlying functionality in MoveIt, and generally in the ROS ecosystem.

Interestingly enough, having worked at MathWorks, which has mature software (MATLAB and Simulink) that has been around for over 30 years, has given me an interesting perspective on what it takes to make a tool that is both powerful and easy to use. It’s tough to strike this balance, but the exciting work to make that happen takes an entire team. As someone who has used, taught, and marketed graphical tools like Simulink and Stateflow for robotics applications, it’s quite cool now doing this in a more specialized fashion and within an open-core business model.

I had a pretty seamless transition into the team in that I first set out to integrate the pick_ik solver into MoveIt Studio. In doing so, I found some extra bugs that the unit tests and interactive testing with RViz did not catch. This was a combination of moving from the more manipulable 7-degree-of-freedom Franka Emika Panda to the 6-degree-of-freedom Universal Robots UR5e, but also in uncovering integration issues with MoveIt Task Constructor for task scripting and MoveIt Servo for endpoint jogging. While it wasn’t the most glamorous task to figure out why the arm was doing weird things, it was great that we got to catch these bugs before our initial release of pick_ik!

I also did some work on our existing behaviors for opening doors. This began with bringing in a new admittance controller to prevent the arm from stopping itself while moving through contact, but also switching away from another workaround to this same issue where the circular arc motion to open the door was discretized into small linear segments. Here, I pulled in an existing circular trajectory generator available in MoveIt. Next, we’re starting to work on bringing perception into the mix for tasks like these, so as to remove the need for users to manually specify motion goals by, for example, clicking an image.

I’ve also had the chance of doing a lot with Docker while working on the MoveIt Studio code base. Basically, as the “new guy” I found it challenging that there wasn’t a standard developer workflow and everyone settled into doing their own thing. After some deliberation, I was tasked with modifying the Docker container we were already using for CI and production to also work for developers. In jumping into this pit of environment variables, install dependencies, build system idiosyncrasies, and making stupendous mistakes, I became quite intimate with Docker Compose. This has inspired me to revisit my use of Docker in my personal projects, which to me signifies a perfect learning cycle!

In general, my PickNik experience has held true to my expectations so far. A lot of ROS, a lot of diving through open-source software, and no shortage of work and variety thereof. The people here are talented, genuinely motivated, and I’m learning so much from them every day. My only mission now is to try and keep up; If I’ve learned so much here in 3 months, I’m excited for what next year brings.

---

Advent of Code + Rust

For those unfamiliar, Advent of Code is an online challenge that started in 2015, which takes the concept of the advent calendar but makes it super nerdy. Specifically, you get a coding puzzle every day between December 1 and December 25 (Christmas Day) that you can implement however you want – you simply have to submit answers to the puzzles to get credit.

Many of my new colleagues were raving about the Rust programming language, and in parallel set up a private Advent of Code leaderboard. Their reviews of Rust were so glowing that I felt compelled to try both these things out in one go with … all the free time I didn’t have. But here we are.

Overall, I found Rust to be a great language and I can see why so many people enjoy it. In my opinion, it took the best of Python and C++ and smashed it together along with some unique properties. I love the safety aspect — that so much is hoisted up to being a compile-time error. While this can be annoying when you’re a beginner and you keep trying to do things that are against the intended usage of Rust, once you get the hang of it it’s very much appreciated… especially if you’ve written C or C++ and know how easy it is to write unsafe code.

On the C++ side: Rust took many of the great features introduced in modern C++, but being a newer project they were able to introduce it into the language with much less baggage. The result is you get a lot of default behavior that has yet to uniformly make it into legacy C++ projects. Here are some examples that stood out to me:

• Data is by default immutable and types are automatically inferred, so for example a let x = 2 in Rust translates to a const auto x = 2 in C++.
• Rust constructs such as Option and Result are ubiquitous to handle unexpected output, in contrast to the equivalent C++ features like std::optional (C++17) and std::expected (C++23?) that are nice-to-have, but in no means enforced… or even present in the standard library at the time of writing this post!

On the Python side: Python may not be the most elegant or fastest language, but it is everywhere because it’s easy to use and tons of people use it. So one thing that Rust did right is that it created a systems language with modern tooling to nurture a similar community of access. Some things here that caught my attention:

• Simple, yet commonplace, things such as printing and debugging are truly simple in Rust. In that aspect, they more closely resemble the ease of development of an interpreted language like Python than they do a systems language like C++.
• The Cargo build and packaging system is very much integrated with the language, unlike the C/C++ tooling which is all over the place for perhaps valid historical reasons.

• The Rust ecosystem seems like it’s growing at a rapid clip, in that I could find “crates” that did everything I needed for Advent of Code.

I must say that Advent of Code was more time consuming and frustrating than I envisioned. While it was great getting to learn Rust and also sharpening my algorithmic chops with challenging puzzles, I conclude that I prefer working on projects with more tangible outcomes. It’s unclear whether I’ll do Advent of Code again, but I certainly hope to use Rust again in the not-so-distant future! Maybe in a less time-sensitive, brain-teaser context that lets me focus more on the language and tooling, where I still have much more to learn.

You can see my detailed notes, and more importantly the code in its 25-day progression from complete n00b to maybe acceptable, in my GitHub repo. Also, if you’re interested in Rust you should read the book and/or go right into the Rust playground.

---

Conclusion

To summarize, my 2022 has been a lot of hard work, and even more coming to terms that our time on this planet is finite and we’ll never get to learn and practice everything we want. Putting the dismal tone aside, I’m satisfied with my achievements for the year, and in the coming year I see no shortage of things to keep trying to write about on here.

Here are my goals for 2023:

• Continue expanding pyrobosim: There is much more to do here. I already mentioned the work I’m doing with my PickNik colleague on integrating a more complex robot as a TAMP demo, but there may be a few other collaborations coming down the pipeline as well. I also hope to keep getting more community involvement as this tool grows. If you’re interested, please reach out!
• More motion planning: I’m intentionally keeping this one generic. No doubt, progress towards this goal will happen organically on the job. However, I also want to try create educational content both on my personal site and for PickNik. It may involve talking about MoveIt, about motion planning in general, or both!
• Take better care of myself: Honestly? I’ve been suffering from increasingly bad impostor syndrome and have been on the edge of burnout for a while now. Especially near the end of this year with the new job and lots of ongoing projects (although many of them self-inflicted), I maybe let myself go too far with screen time. This is a public self-callout to get back on track with my running, as mental and physical health pays dividends, but more importantly I figured it was worth sharing with everyone as a reminder that you’re not alone in the struggle. Robotics is hard, and life is harder.

Thanks for reading, and I wish you all a fantastic 2023. Push yourselves to learn something new and do great things, but please don’t do it at the expense of your well-being.