Python and Jupyter – a 21st century data stack for business users, not just IT!

Engineering

Python and Jupyter – a 21st century data stack for business users, not just IT!

Stephen LeMasney

Stephen LeMasney

20/07/22

At FINBOURNE, we’re increasingly hearing from investment managers who are looking to upskill in technologies that will allow them to do more with their data, and ultimately glean more value from it.

Take Portfolio Managers, for example. High on your priority list is access to market data, to test alpha-generating data science theories, whereas if you are a Risk Manager, you want to automate performance reports, which are currently manually built in Excel each morning.  To achieve these goals, you’ll want a robust data platform. But you’ll also want flexible tools which sit on top of that platform to interrogate and analyse the valuable data within. We’ve landed on Python, paired with Jupyter notebooks, a combination we feel delivers controlled, plug-in access to rich investment data sets and heaps more benefits. And here’s why…

What is a Jupyter notebook?

In one sense, you can think of a Jupyter notebook as a “turbo-charged” word processor, which allows you to run interactive Python (and other programming languages) alongside diagrams and text. If you are a portfolio manager, running some scenario analysis on your portfolio, you can run real-time data analysis with Jupyter – pulling data from multiple sources. Combined with narrative text and automated diagrams, you can now explain your investment thesis with far more clarity. This notebook can be shared with others who can reproduce your findings in one or two clicks, making it more interactive and easier to make your investment case. What’s more, these notebooks can be version controlled in a repository like GitHub, so you can maintain control over document lineage. They can also be automated with a scheduling technology, reducing the need for manual processes.

Why Python?

These notebooks work with many programming languages (R, Julia, etc) but we’re big fans of Python, especially for business users. Why? Firstly, Python is what’s known as a ‘higher-level’ language. This means that a lot of the complexity is hidden away and the language syntax is easier to understand. A business user who currently uses SQL and VBA might find that Python is a natural next step. Secondly, it has huge community support – Python has been the ‘most wanted’ language on the Stack Overflow Developer Survey for the past 5 years running. For this reason, it is supported with plenty of online documentation to help you solve problems quickly. Finally, it has a very rich ecosystem of open-source data tools. Pandas is hugely popular in the financial services industry for spreadsheet-type functions (and more). While, tools like TensorFlow and sklearn offer ML capabilities and Seaborn empowers you to create rich visuals of your results.

What if I am not a developer?

The Python-Jupyter stack is a great tool for business users who want to learn new data skills. You don’t need to be a seasoned software engineer to get started. Firstly, most notebook implementations come pre-installed with all the packages you need to write useful code quickly. If you are a Risk Manager who needs to get on-demand holdings for your portfolios, you don’t want to spend 45 minutes resolving package dependencies and interpreter issues. The notebook and installed vendor packages should just work “out-of-the-box” . Secondly, as mentioned above, the Python syntax is very intuitive and readable. To illustrate, we can guess that the code below is requesting holdings for a portfolio called “UkEquityPortfolio” for holding date 1st of July 2022.

That said, the Python language is also extremely powerful. It supports all the features you’d expect from a modern, professional programming language. The beauty of it is that while the first barrier to entry is low, you can take it to the next level when you’re ready.

How do I integrate this stack into my organisation? 

To get the most value from the Python-Jupyter stack, you’ll want to place it on top of a robust investment data management platform. Ideally the platform will have a full set of entitlements with audit capabilities. The ‘get holdings’ request made via the notebook should be subject to the same set of entitlements as a ‘get holding’ request made from a connected PMS. Additionally, you’ll want the notebook to have full integration to this underlying platform. Usually this means a Software Development Kit (SDK) which handles the boring (but important) stuff – authentication, converting API requests to objects in your preferred programming language, data validation etc. A good SDK means that you can spend more time solving business problems and less time writing ancillary code to keep everything glued together.

Why is any of this important for me?

At this point, it’s worth stepping back to see what forces are driving the desire for these technologies. On the one hand, there is a push for automation and efficiency. Almost any manual/repetitive task, with fixed rules, can be automated with something like Python (assuming you can get programmatic access to the underlying data). On the other hand, investment managers want to ‘mine data’ to get better insights. For example, if you are a trade operations team, you might want to see which brokers have the most trade confirmation failures this month, and why? While a portfolio manager might want to explore whether a new alternative data set has any alpha opportunity. In previous years, these tasks would have been the sole responsibility of IT teams. More recently, however, we are seeing a blurring of these lines. In a survey last year entitled The Future of Asset Management (2021) Accenture reported that over half of respondents saw “IT and the business moving closer together… with skillsets overlapping”.

Try it for yourself

All LUSID domains come pre-installed with a Jupyter instance. Browse to www.<YOUR-DOMAIN>.lusid.com/jupyter to launch the tool (you’ll need to create a LUSID account if you don’t already have one). There you will see a set of notebooks from our sample-notebooks project covering a wide variety of use-cases ranging from running valuations on your portfolio to measuring performance. The notebooks can be run out-of-the-box. There is no setup needed and authentication happens automatically via your browser (if you have already logged into LUSID).

Try out the notebooks for yourself and let us know if you have any questions at talk@finbourne.com 

avatar

Stephen LeMasney

20/07/22

twitterlinkedinfacebook

Related articles

De-risking operational change in capital markets

Stephen Collie10/11/22

Delivering safer, better and faster data with the Modern Financial Data Stack

Finbourne07/11/22

Consolidated Tape: One size does not fit all

Neil Ryan25/10/22

Consolidated Tape: Unlocking the value of CT data to optimally assess liquidity

Neil Ryan25/10/22