Development and navigation in a large codebase can prove quite the challenge. A common tool when tackling such a large codebase is an Integrated Development Environment (IDE). Unfortunately, the main programming language at Booking.com, Perl, has limited IDE support. This is the story of the joint effort with the open source community to address this need of IDE support for the Perl programming language.
The Search
In the past I had worked on large projects in a Java-based environment and learned the importance and the value of a good IDE supporting you in your daily tasks, while helping you navigate your way through the endless lines of code in your projects. When I first started to work at Booking.com (about 2 years ago), I was new to Perl and began searching for an IDE for Perl.
I started with Perl oriented IDEs and moved to plugins for Eclipse and IntelliJ, but these attempts came up empty.
Most of the solutions only provided syntax highlighting while others suggested basic text-based or token-based autocompletion. None of them provided full integration in which the IDE understands the relationships between variables, subroutines, and their arguments and packages. I also wanted support for patterns like inheritance.
For a while I resorted to working with Sublime Text but even Sublime Text was simply an enhanced text editor rather than a complete IDE.
Searching for a package is a breeze with the 'Navigate to Class' feature
The Seeding Grounds for Innovation
Luckily at Booking.com we have hackathons (spanning across multiple days). They allow us to experiment with ideas and projects. They enable the business to explore various initiatives which might include adding new tools to making the workflow more effective, exploring new systems or tackling any of the issues for which you normally don't have time in your daily work. I took this opportunity to start working on a proper plugin to provide us with a full IDE for Perl.
With referencing: you can click on a package and open it's file or click on a subroutine call to go to it's declaration
Starting from Scratch
While I encountered some discussions regarding the challenges of parsing perl or expressing it withBNF (a notation technique often used to describe syntax for programming languages), I knew that with enough resourcefulness the majority of use cases can be covered with regular expressions.
Since I was familiar with IntelliJ's capabilities and its plugin development workflow, I decided to develop a plugin that will integrate into the custom languages support, that will make up for the limited support it offers Perl right off the shelf.
One of the challenges when parsing a language (especially a dynamic one like Perl) is that it requires a lot of iterations. Every time you add one thing, it breaks two other things. However, after several hackathons and continuing to work on the plugin at home, I managed to cover most use cases.
I also added many important features like:
- Object Oriented autocompletion for packages (including inherited subroutines from parent packages)
- "Go to" implementation of a package by clicking on an icon in its row
- Searching by package name
- Syntax highlighting
- Creation of Perl project/module and SDK selection
- Support for running
.plfiles - Subroutines autocompletion with arguments
Scalars, subroutines (and their arguments), packages and inheritance packages - all get autocompleted with the Perl intelliJ Plugin
First Impressions
I showed the plugin around the company and several developers started using the plugin and contributing to the project. This led to a substantially increased development velocity for the plugin, which enabled adding more functionality to the project.
As time passed, I started to realise that without a full BNF there will always be some functionalities in IntelliJ that the plugin could never support (creating references between the various components of the language, syntax error checks, etc.).
I also knew that the more this project would grow, it would need more time and work than a single person could provide. Knowing Booking.com supports employees sharing and contributing to the open source community, I uploaded the project to GitHub and began to roam the IntelliJ plugin forums.
MySQL queries autocompletes databases, tables, and columns
Working with the Open Source Community
Shortly after releasing the plugin, I was contacted by Alexandr Evstigneev, a talented developer from the open source community, who had already created a similar plugin.
Since Alexandr managed to express nearly the entire Perl language in BNF in his plugin and the Booking.com plugin already contained so many features, we knew we could benefit from merging our projects. I also saw this as a great opportunity to give back to the open source community and started migrating our code to his repository, feature by feature.
After a couple of months of integration, along with more people contributing from the open source community, we got IntelliJ to work as a fully pledged Perl IDE that understands Perl as a language. Referencing, Find by Usage, Going to subroutine declaration, Database Integration and autocompletion, Package Hierarchy - these (and more) are all part of the support this plugin provides from within IntelliJ.
While Alexandr continued to work heavily on the BNF structure and adding features with other contributors that joined the GitHub project, I presented the plugin at an AmsterdamX.pm Perl Mongers group meetup for English-speaking Perl developers in Amsterdam and The Netherlands. The presentation generated great feedback and excitement that motivated us to continue our work on the plugin. You can install the plugin directly through IntelliJ's plugin menu (look for Perl5 Support).
the GitHub project of the plugin
Conclusion
Today, the project belongs to the community and is managed by Alexandr, the main contributor. The work on this project during our hackathons with the open source community has been fruitful and beneficial both for us at Booking.com and for the community as a whole.
Looking ahead, there are still more features to be added like Mason support, enabling the client debugging and server debugging. So if you're interested in the plugin, you're welcomed to join the development effort at GitHub!