MIST

by David Price

At the time of my PhD application the placement in industry that was tagged on to the process sounded like a good idea, I thought to myself that it would provide the opportunity to see what other career options my Master’s degree offered me and hopefully illuminate which of the two paths, academia or industry, would better suit me at the end of my studies. However when the time came to start looking around and applying for opportunities, late in my 2^nd year, I was angry at my past self: ‘why couldn’t you have just chosen a normal PhD so you wouldn’t have to worry about all this?’

Now that the prospect was actually upon me, and not just a seemingly good idea for my CV a few years ago, the foray into the world of industry conjured imagery of an Office Space–esque hellscape of grey cubicles, excessive bureaucracy and miserable excel zombies. However, my experience at Snowflake Software, at which I spent the second half of last year, turned out to be a far cry from this “potentially inaccurate” mental image and as a result I have entirely new perspective on life outside of academia. I was introduced to Snowflake via the GRADnet Employer Engagement scheme, ran by the South East Physics Network (SEPnet), who host regular training events and distribute placement and graduate opportunities to participants. If you are a member of one of the participating Universities and are interested in also doing some kind of placement I would recommend checking their website out (I’m also sure there will be similar resources for students at non-participating universities, just ask!). Unfortunately for readers of this post I am unable to go into extensive details of the work I was doing at Snowflake (for intellectual property reasons), so instead I will focus mostly on my thoughts and reactions to the experiences I had whilst working there.

Day to Day

Snowflake Software is a small software company, conveniently located in the centre of Southampton, that specialises in the aggregation and redistribution of flight, weather and aeronautical data from a variety of sources. During my placement I was attached as a Data Scientist to the IATA (International Air Transport Association) Turbulence Aware project. Snowflake had landed a competitive deal to build, host and manage IATA’s cloud-based turbulence data platform which now receives live in-situ turbulence data from thousands of flights all over the world. On my first day I was given access to millions of individual data points, each containing a measured value of turbulence, location, time, and many other meteorological variables over a timespan of approximately one year. The brief was simple: ‘see what you can find out about this dataset’ (paraphrased), and so began 5 months of me exercising a mild obsession with maps by plotting and producing just about any kind of interesting analysis I could come up with. This open ended kind of approach to the placement was both familiar and enjoyable and I was given free rein to choose what software and coding language I wanted to use to approach this interesting dataset.

As a result the actual work was similar in many ways to some of my work during the PhD, the vast majority of my time was spent writing python code, and I would liken it specifically to the final stages of analysis on any data set you could have acquired during your research. However, the day to day experience was different in many ways to that I have experienced in academia. I would obviously put most of the differences down to the simple fact that Snowflake was delivering a product for a paying client, and as such they did not have the flexibility and freedom on timescales that we sometimes are party to during our PhD. Snowflake practices a development methodology known as ‘Agile’ which utilises an iterative development process, with a focus on collaboration with the client on a regular basis. Starting from day one I was folded into this system, but since I was not a software developer I was sort of operating on the fringes of this methodology - in short I contributed to the agile development process in two ways:

1. Participation in the daily ‘stand up’ sessions with the team. A stand up essentially consists of each member of the team giving an update on the tasks they have accomplished since the last meeting, what they hope to accomplish before the next and how they are feeling on that day. Whilst for the others it was useful for keeping track of who is developing what aspect of the software at any time, for me it usually consisted of: “spent most of my time wrestling with matplotlib yesterday, I imagine I will also be doing that for the most of today” an ‘experience’ most PhD students are already intensely familiar with.
2. Participation in the bi-weekly ‘sprint review’ sessions with the client. These sessions were a much more formal affair, in which the team hosted a conference call with the IATA turbulence aware group. These meetings consisted of a 20-30 minute presentation in which any new software features were demonstrated by a member of the team and appropriate feedback was given and questions asked. After this it was my turn to take over the presentation and talk the IATA team through the analysis I had done during that period and take suggestions for further research directions. 

The idea behind both of these processes is to consistently receive feedback and guidance, which in turn helps to steer your day to day work in the direction the client desires. This meant that I spent very little time wandering what I should be doing as I always had one or two fresh suggestions of things to look into from the IATA team. Of particular note were the inputs from the enthusiastic subject matter expert, ex-pilot Bret King. Bret provided an unending supply of anecdotes from his extensive career as well as incredibly useful insight into the kind of turbulence data and information actual pilots really wanted and could use effectively.

Overall I think the Agile system has both pros and cons. Initially I was frustrated by the extremely regular meetings at the start of the day (particularly when I didn’t accomplish much in the 24 hours period) but when you are working towards a client’s expectations I do think it is useful to be made very aware of the current state of progress. One of the things I found most difficult about Agile was having to assign a point like system to your tasks wherein you make a prediction on how long something is going to take you. I know I have a particularly bad habit of telling my PhD supervisor that something will be ‘easy enough’ and that ‘I’ll have it ready by next week’ only for this prediction to fall completely flat. I think the Agile system is also susceptible to this error in judgement and often in the less flexible environment of industry I saw it leading to mad rushes before a deadline to deliver all that was promised. 

Extra-curricular activities

My time at Snowflake didn’t just consist of coding and meetings with the clients, in fact there was a lot of what I have called extra-curricular activities on top of all the rest. I would be remiss if I didn’t mentioned the many games of ping pong I played at lunch, the end of the day, and pretty much at any time I could convince another office member to take five minutes off. In fact a personal favourite outcome from my placement is the development of my forehand smash, it’s absolutely lethal. On top of this, those who know me will know just how sickeningly competitive I am which meant that obviously I kept an extensive spreadsheet of my win loss ratio against everyone in the office, the result of which you can see below. The ‘BIG 3’ marker is a combination of the companies’ best players (pretty much the leadership team), whom I failed to take a single game off, much to my intense dissatisfaction. You’ll also note I purposely left out the total number of games played…

On another more serious note, the true highlight of my placement was the incredible opportunity to travel to Chicago and present the work I had been doing. Approximately three weeks into my placement I was given a chance to attend the IATA Turbulence Aware forum. The forum was a conference like congregation in which representatives from airlines all over the world gathered to discuss the applications of the new turbulence dataset IATA was in the process of making available to them. My presentation was to be the last of the two day itinerary – a 45 minute slot in front of a panel of meteorologists from the likes of Delta, United and Southwest airlines – to discuss how data science can help direct the most efficient use of this data to the airlines. This was an incredibly useful, albeit mildly terrifying, experience for me. The differences between an industrial conference setting and those in academia are not very large, in fact it was much of the same; small talk over lunch, hurried technical conversations between talks and a few pints in the pub afterwards. One noticeable difference, for me at least, was that I was given an optimistically large stack of business cards to disseminate between attendees, and in return I ended up with a surprisingly colourful stack of others for myself. Finally, out of fear of this becoming a travel blog, I will not say too much about the time I got to spend exploring the city of Chicago. I will simply say that it is a cool city and I would highly recommend a drink or two along the canal/riverwalk if you ever get the chance!

Final Note

All in all I would definitely say I had a positive experience on my industrial placement working with Snowflake Software. It turned out to be an extremely eye opening and instructive five months - a lot of the skills I have developed during the course of my degree and PhD proved not only to be useful but also extremely valuable and interesting to a variety of companies. On top of that I was introduced to new experiences and opportunities to develop skills working within a larger team and operating in a more formal manner. I would recommend!

To find out more about Snowflake Software then check out their website here: https://snowflakesoftware.com

If you would like to find out more about David’s experience with an industrial placement then you can This email address is being protected from spambots. You need JavaScript enabled to view it..