How data scientists can achieve maximum impact in a federated organization

As a data scientist your efforts can be dramatically impacted by the way the organization is structured. This is particularly true of federated organizations where executive offices — such as data science departments — are unable to mandate policy but instead have to win over others in the organization. For this reason, it is just as important to effectively navigate the organization's political landscape as it is in developing deep analytic insights. Finding ways to prove to others the importance of your work in a federated organization ensures that your work will have a long and substantial impact.

What is a Federated Organization?

A federated organization is one where the executives don’t have direct control over the operational offices. Instead, the operational units determine for themselves how best to operate. Take for example school districts. Their district office primarily plays a support role, leaving it up to the schools how best to manage their daily operations. Large companies are also typically federated. My current company is structured so that local districts call most of the shots, while the various corporate offices like finance, human resources and predictive analytics support those operations.

In a federated organization data science departments cannot rely on executive fiat to ensure their insights and tools are adopted by others in the organization. In this type of organization, the executive’s ability to mandate is dramatically reduced as the operational offices have more control in how they operate. As a result, data scientists’ impact is predicated on their ability to influence others in the organization. They will need to convince operational stake holders to adopt their data driven insights rather than just rely on executive mandates. Failure to win over the schools or districts that run an organization will inevitably result in their work being ignored and their impact being non-existent.

This happened to me a fair amount while working for a school district. Analytic reports we spent months developing for a general audience never saw the light of day. Or tools we used to help predict when students would drop out were never utilized. Even in a corporate environment I have seen my proposal on supplier segmentation ignored when it didn’t gain traction within the supply chain department.

Ways a Federation Can Hinder Data Science

Data scientists working in the executive office can have their impact minimized in at least four different ways when others in the organization ignore their work.

Data Driven Decision Making- In a federated organization decision makers are largely free to choose when and which data insights they should incorporate into their operations. In my current position, this is commonly manifested in the form of people in the field making operational decisions based on their intuitions. This occurs frequently around job bidding. We will develop solid predictive models around the optimal bid using data science and historical data — only to have it ignored because the decision makers ‘feel’ that our number won’t win the job. If our predicted bids are accurate, ignoring our insights will cost the company money in the form of losing job opportunities. Or if we do win the job, we will end up losing money by offering a bid amount lower than what the client was willing to pay.

Collecting Data- Vital to data focused organizations is collecting large amounts of high quality data. Improving the data structure and quality of the organization enables data scientists to build better models and have greater confidence in their findings. Improving data typically requires that operational members put more effort into collecting high quality data and developing the infrastructure necessary to store and utilize this data. When they don’t see the value of high quality data, they make data scientists’ jobs more difficult and limit their organizational impact.

Experimenting- Data scientists oftentimes will need to test the effectiveness of different policies to help drive decision making. When I worked for a school district I setup an experiment to test the effectiveness of several different teacher training programs. Success in these experiments would require considerable effort from operational personnel. When they are uninterested in the results the experiments execution can suffer, causing the experiment to fail.

Developing Analytic Tools- Almost always a data scientist’s work will need to be developed into something usable by others in the organization. These individuals can decide for themselves the extent to which they use your tools and insights. When they choose not to use your work it reduces your impact on the organization.

I saw this first hand when I was in the public schools. Our data strategic department would develop excellent visualizations to help teachers understand their students’ math and reading performance. However, the majority of the teachers simply ignored these resources, thus minimizing our impact on improving students’ academic outcomes.

Having an impact in a federated organization

Clearly, federated organizations can hinder you ability to have an impact as a data scientist. However, I have learned some techniques to improve your impact in these organizations. It starts by understanding that in federated organizations other people don’t have to listen to your insights or use your tools. Instead, you have to get others excited to use your analytic reports, and demonstrate that your tools will improve their work efficiency. Effectively, you need to get buy in from as many people in the organization as possible. The more people convinced of the value of data science, the more impact your skills will have on the organization.

Below are a some ways that I have found to help others find value in your work.

Awareness- In larger organizations people are often unaware that there is a data science department. Worse still, they have no idea what data science is or how it can help them. Obviously this will substantially reduce your impact on the organization. To remedy this, data science departments should find ways to become more visible within their organization by educating staff on the kind of work they do and how that can help them get the job done.

When I worked for the school district, our department developed something of a data science seminar. We would invite people across the district, enticing them with candy and cookies and presenting our work to them. We also broke into small groups to give our coworkers time to discuss some of their data problems, and to learn how we might be able to help. At first, I did not view these sessions as a good use of my time, but eventually I realized they were an excellent way to boost our awareness within the organization.

Low Hanging Fruit- Since data science departments are new to most organizations, there are likely to be low hanging fruit. Be on the look out for opportunities within operational departments that require minimal amount of work, but will have a dramatic impact for them. Doing so increases your visibility, helps them understand the work you do, and guarantees your work will have an impact.

A recent low hanging fruit I worked on was improving a survey for a group charged with understanding the company’s relationship with its suppliers. To do so, they had sent out a relatively lengthy questionnaire to a bunch of our suppliers. Reviewing the survey, I could see redundancy in the questions they drafted, so I spent a day using an exploratory factor analysis to identify where we could reduce the size of the survey. Even though the analysis took less than a day, the output saved time for both the team and suppliers.

Develop Allies — In improving your impact it’s important to identify people aligned with incorporating data science into the organization. There are at least two kinds of allies you want to be on the look out for. The first kind typically has a supervisory or director level role, and is well positioned to influence the organization and is excited about data science. Make sure you work with these individuals and keep them happy because they will become a major source of your projects. Also, providing them analytic support will ensure they become vocal advocates for your work by bringing awareness to your department even when you are not around.

The other kind of ally is one that has access to resources you need to be more effective in your job. For example, individuals in the IT department that have access to data or data infrastructure. Even though they may not have a director level role in the company, developing relationships with them is essential to producing high quality analytic work. Creating several allies within this group ensures that you can get access to the data you need and that the infrastructure necessary for doing your work will be built and maintained.

Developing these relationships can be as easy as meeting them in person or bringing them some tasty doughnuts. The point is to meet face to face so they will see you as an amiable person, so that when you ask for help they are eager to respond. Also ensure the relationship isn’t one sided by making sure to offer help when they need it, especially when a data science solution can solve their problem.

Feedback- When developing dashboards and other analytic tools intended for use by others in an organization it is vital that you solicit feedback from them. Without their feedback, you will likely build tools that no one is asking for and will therefore go unused. Make sure to get input from anyone that could possibly use the tool. The more involved people are in the development, the more likely they will use the final product.

Both in my current role and when I was at the school district, we fastidiously sought feedback on the analytic tools we were developing. In addition, we would typically have an iterative cycle of development, allowing the tool to develop as people in the organization gained better understanding of what was being built.

Know your limitations — I recently learned a lesson on the importance knowing my limitations. Frustrated with the speed of project, I decided to more aggressively manage the project. Rather than being appreciated for my extra effort, I found myself in the middle of an organizational turf war. I quickly realized that I was not an experienced project manager and that my efforts would be better served just providing data science support to the project.

It is helpful to remember that data scientists are brought into an organization for a specific set of skills. Being good at machine learning doesn’t always translate into being great at other skills. Be humble enough to your know your limitations and focus on using the skills the organization hired your for.

Concluding Remarks

Federated organizations can make it challenging for data scientists to have an impact. You will typically be unable to rely on executive fiat to ensure your insights and tools are utilized. Instead, the best way to have an impact on the organization is by raising awareness through developing the insights and tools that improve others ability to work. By helping them realize how data science can improve their work, you will maximize your own impact within that organization.

Related posts:

Every year new mothers take their own lives because motherhood is nothing like they were promised. Although postpartum depression and anxiety is more openly discussed lately, there are still many…

Not that we should never eat anything unhealthy, skip a workout, spend a day lounging around on a beach, or on the couch, but that it should be a treat. Instead of moderation we should aim for doing…

Have you ever wondered how we get hooked to certain websites and spend hours without realising? Or, how it becomes so easy to navigate between some websites while with others it’s simply annoying…