You have to admit, seeing a comment like “This is super clean 😎” or “Didn’t know this could be done this way” on your code or pull request fills you with a wonderful feeling. Personal experience has taught me, embracing good software engineering principles and making the most of the existing language functionality is the recipe for good code that others will feel grateful for.
As an MLE I use Python day in and day out. Python is a great option for ML practitioners due to its low barrier to entry combined with the massive ecosystem of scientific tooling.
This means that, an individual with little to no software engineering knowledge can quickly start using Python.
This last statement can be said in two different tones of voice; positive or negative (try it!).
It may look like it’s a blessing at first, but in the grand scheme of things, the lack of the confinements of software engineering principles (e.g. Types, Objects) dissuade the engineers (MLE) or scientists (DS/AS) from writing good code (Trust me, we already got a bad reputation among software engineers as not-so-good engineers). This inevitably leads to unreadable, unmaintainable and untestable spaghetti code in most cases. And worse, one-day it becomes some unsuspecting victim’s worst nightmare to reuse this evil code. And you may even see a domino effect, where code built on top of bad code leads to … more bad code. Ultimately, this could even lead to organizational headaches down the track.
The bottom line is, doing something in Python is easy, but doing something the right way in Python is difficult. After 8+ years grappling with Python, I’m still learning different (and better) ways to improve my code. I’ve been blessed with good software engineers that would constructively criticize my code, when I do things in inefficient manner. Count your blessings if you have the same support. Here, I’m going to share a few levers you can pull to take your Python skills to the next level.