Raja Iqbal

An autonomous AI agent broke into Hugging Face’s production infrastructure over a single weekend, executing over 17,000 recorded actions, harvesting cloud credentials, and moving laterally across internal clusters without a single human directing it.

The Hugging Face security breach 2026, disclosed publicly on July 16, 2026, is the first publicly confirmed fully autonomous AI agent intrusion against a major tech company.

In this post, you’ll learn exactly how the attack worked, what was compromised, how Hugging Face detected and responded to it, and what your team needs to do right now to avoid becoming the next target.

What Happened: The Hugging Face Security Breach 2026

The attack unfolded silently before July 16, 2026, completing its entire run over a single weekend. When Hugging Face disclosed the breach, the security community had to reckon with something genuinely new: not a human hacker using AI tools, but an autonomous agent operating independently from start to finish.

Here’s the timeline at a glance:

The scale and speed of the operation set it apart from every prior AI-assisted attack. And just to make it clear, this was not a human using an AI copilot. It was an agent running autonomously, adapting, and persisting.

How the Attack Worked Step by Step

Understanding the mechanics of this AI agent cyberattack is critical for any team building on or with AI platforms. The attack chain was elegant and ruthless.

1. Weaponized dataset upload

The attacker uploaded a malicious dataset to Hugging Face’s platform. On the surface, it looked like any other community contribution. Underneath, it was a carefully crafted payload.

2. Remote-code dataset loader exploitation

Hugging Face’s data processing pipeline supports custom dataset loaders, code that runs automatically when a dataset is processed. The malicious dataset exploited this feature to execute arbitrary code on a processing worker. they invite the world to upload code-like artifacts, and processing those artifacts means executing someone else’s code on your own infrastructure. Our deep-dive on Prompt Injection & Claude Computer Use: 2026 Guide covers the mechanics of this attack class in detail.

3. Template injection in dataset configuration

Simultaneously, the attacker exploited a template injection flaw in the dataset configuration, feeding unsanitised input into a template engine to run code.

4. Worker-to-node privilege escalation

Initial code execution landed on a processing worker — a lower-privilege environment. The agent then escalated to node-level access, gaining a much broader foothold in the infrastructure.

5. Credential harvesting

With node-level access, the agent harvested cloud credentials and cluster credentials stored in the environment — executed at machine speed with no human pausing to make decisions.

6. Lateral movement across internal clusters

Using the harvested credentials, the agent moved laterally into several internal clusters. Each hop expanded the blast radius. The entire lateral movement phase happened autonomously.

7. Self-migrating C2 infrastructure

Throughout the operation, the agent maintained command-and-control (C2) infrastructure staged on public services and that infrastructure self-migrated to avoid detection. This is a level of operational sophistication that, until now, required a skilled human operator. The attack surface that made this possible is exactly what we explored in The State of MCP Security in 2025.

And the full operation completed over a single weekend.

The AI vs. AI Detection Story

Here’s where the story gets genuinely fascinating. Hugging Face detected this intrusion not through traditional SIEM rules or human analysts, but through its own LLM-based anomaly detection pipeline. An AI caught an AI.

But the forensics phase revealed an uncomfortable irony. When Hugging Face’s security team attempted to use a commercial frontier AI model to assist with incident analysis, the model’s safety guardrails blocked the forensic work. Queries involving malicious code, attack patterns, and exploit chains triggered content policies designed to prevent misuse.

The team fell back to a self-hosted open-weight model: GLM 5.2. Running it on their own infrastructure meant no data left their environment, no third-party safety filters blocked legitimate security research, and the team retained full control over the analysis.

This creates a real operational gap for security teams doing legitimate forensic work which self-hosted models fill.

The broader implication: AI raises the stakes for security precisely because it enables automation and scale — attackers can now use AI to launch wider, faster vulnerability searches than any human team could match. The defenders who catch those attacks will increasingly be AI systems too.

What Was (and Wasn’t) Compromised

The attacker’s focus was on credentials and internal data, the kind of access that enables further lateral movement or future attacks. Public-facing assets that millions of developers depend on were not tampered with. That’s a meaningful distinction, but it doesn’t reduce the severity. Harvested credentials are a launchpad, not an endpoint.

How Hugging Face Responded

Hugging Face’s response was methodical and transparent:

• Patched both vulnerabilities: the remote-code loader exploit and the template injection flaw were both closed
• Rebuilt compromised nodes from scratch: rather than attempting to clean affected nodes, they wiped and rebuilt them entirely
• Revoked and rotated all affected credentials: every harvested token was invalidated and reissued
• Deployed stricter admission controls: new controls limit what can be executed in the data processing pipeline
• Improved detection response time: future anomalies now page responders within minutes
• Reported to law enforcement: the incident was formally reported to relevant authorities

This response reflects a mature incident response posture. The speed of the rebuild and the transparency of the disclosure set a reasonable standard for the industry.

Why This Breach Changes Everything

The Hugging Face security breach 2026 isn’t just a story about one company. It’s a signal flare for the entire AI industry.

The data pipeline is now an attack surface. Every AI platform that accepts user-contributed datasets, models, or code artifacts is running a supply chain and that supply chain can be weaponized.

The attacker-defender asymmetry just got worse. A human attacker is constrained by time, attention, and fatigue. An autonomous AI agent is not. It executed 17,000+ actions over a weekend. A human security team cannot match that pace without their own AI-powered defenses.

Self-migrating C2 infrastructure is a new baseline threat. Traditional IOC-based blocking is insufficient when the attacker’s infrastructure moves autonomously. You need behavioral detection, not just signature matching.

Attribution is getting harder. When attacks are executed by autonomous agents, the human fingerprints that enable attribution become harder to find.

How to Secure Your AI Infrastructure Now

This breach is a checklist moment. Here’s where to start:

1. Sandbox your AI agents. Any agent that can execute code needs to run in an isolated environment. Our tutorial on LLM Agent Security: Running AI Agents Safely with Docker Sandboxes walks through exactly how to implement this.

2. Treat every dataset as untrusted input. If your pipeline executes code from user-contributed datasets, you need strict validation, sandboxed execution environments, and content scanning before any processing occurs.

3. Audit your template rendering pipeline. Template injection is a well-understood vulnerability class. Audit every place in your stack where user-controlled data touches a template engine. Sanitize inputs. Use logic-less templates where possible.

4. Implement LLM-based anomaly detection. Rule-based detection alone won’t catch novel agent behavior.

5. Rotate credentials on a schedule. Short-lived credentials, regular rotation, and least-privilege access policies reduce the window of exposure dramatically.

6. Build a governance framework before you need it. The teams that responded fastest had governance structures already in place. Our AI Governance Checklist for CTOs, CIOs, and AI Teams and the broader AI Governance Guide give you the frameworks to build that structure now.

7. Evaluate purpose-built AI security tooling. Teams deploying AI agents need tooling designed for that context. Ejento.ai is worth evaluating if you’re looking for AI governance and security tooling built specifically for autonomous agent deployments — covering policy enforcement, audit trails, and agent behavior monitoring.

Frequently Asked Questions

How did the Hugging Face breach happen?

The attacker uploaded a malicious dataset that exploited two vulnerabilities: a remote-code dataset loader that executed arbitrary code on a processing worker, and a template injection flaw in dataset configuration. An autonomous AI agent then used that initial foothold to escalate privileges, harvest credentials, and move laterally across internal clusters — all over a single weekend.

What is an autonomous AI agent attack, and why is this one different?

An autonomous AI agent attack is an intrusion executed by an AI system operating without human direction — planning, adapting, and persisting on its own. This breach is the first publicly confirmed instance of a fully autonomous AI agent successfully compromising a major tech company’s production infrastructure, executing over 17,000 recorded actions across a single weekend.

What data was stolen in the Hugging Face security breach 2026?

A limited set of internal datasets and service credentials/tokens were compromised. Public models, Spaces, and container images were not affected. The stolen credentials represent the primary risk, as they could enable further unauthorized access if not immediately rotated — which Hugging Face did as part of its response.

How to secure AI data pipelines against this type of attack?

Treat all user-contributed datasets as untrusted input — validate and sandbox before execution. Audit your template rendering pipeline for injection vulnerabilities. Implement LLM-based anomaly detection for behavioral monitoring. Enforce least-privilege access and rotate credentials regularly. Sandbox any AI agents that execute code using isolated environments like Docker containers.

Who was behind the Hugging Face attack?

Attribution remains unconfirmed. Sysdig documented a threat group called JADEPUFFER conducting agentic ransomware operations in July 2026, which may be related, but no formal attribution has been made. The autonomous nature of the attack makes traditional attribution harder, as human behavioral fingerprints are largely absent from the attack chain.

Could this happen to other AI platforms?

Yes. Any platform that accepts user-contributed code-like artifacts — datasets with custom loaders, model files, configuration templates — and processes them automatically has a similar attack surface. The specific vulnerabilities exploited here have been patched, but the underlying risk class (supply chain execution via user-contributed content) applies broadly across the AI ecosystem.

Conclusion: The Attack That Rewrote the Threat Model

The Hugging Face security breach 2026 didn’t just expose two vulnerabilities in one platform. It demonstrated that autonomous AI agents are now operational attack tools capable of executing thousands of actions, adapting their infrastructure, and completing a full intrusion cycle without human direction.

The good news: Hugging Face’s own AI-powered detection caught it. The response was fast, transparent, and thorough.

The harder truth: most organizations are not running LLM-based anomaly detection. Most have not sandboxed their AI agents. Most have not audited their data pipelines for template injection. Most do not have an AI governance framework that accounts for autonomous agent threats.

This is the case study your security team needs to read before your own incident response plan gets tested. Start with your data pipeline. Sandbox your agents. Build your governance layer now.

If you’re building with AI agents and need a purpose-built platform for governance, policy enforcement, and agent behavior monitoring, Ejento.ai is designed for exactly this moment — when the stakes of getting AI security wrong have never been higher.

This RHadoop tutorial resamples from a large data set in parallel. This blog is designed for beginners.

How-to: RHadoop (with R on Hadoop) to resample from a large data set 

Reposted from Cloudera blog.

Internet-scale datasets present a unique challenge to traditional machine-learning techniques, such as fitting random forests or “bagging.” To fit a classifier to a large data set, it’s common to generate many smaller data sets derived from the initial large data set (i.e. resampling). There are two reasons for this:

1. Large data sets typically live in a cluster, so any operations should have some level of parallelism. Separate models fit on separate nodes that contain different subsets of the initial data.
2. Even if you could use the entire initial data set to fit a single model, it turns out that ensemble methods, where you fit multiple smaller models using subsets of the data, generally outperform single models. Indeed, fitting a single model with 100M data points can perform worse than fitting just a few models with 10M data points each (so less total data outperforms more total data; e.g. see this paper).

Furthermore, bootstrapping is another popular method that randomly chops up an initial data set to characterize distributions of statistics and also to build ensembles of classifiers (e.g., bagging). Parallelizing bootstrap sampling or ensemble learning can provide significant performance gains even when your data set is not so large that it must live in a cluster. The gains from purely parallelizing the random number generation are still significant.

Sampling with replacement

Sampling-with-replacement is the most popular method for sampling from the initial data set to produce a collection of samples for model fitting. This method is equivalent to sampling from a multinomial distribution where the probability of selecting any individual input data point is uniform over the entire data set.

Unfortunately, it is not possible to sample from a multinomial distribution across a cluster without using some kind of communication between the nodes (i.e., sampling from a multinomial is not embarrassingly parallel). But do not despair: we can approximate a multinomial distribution by sampling from an identical Poisson distribution on each input data point independently, lending itself to an embarrassingly parallel implementation.

Below, we will show you how to implement such a Poisson approximation to enable you to train a random forest on an enormous data set. As a bonus, we’ll be implementing it in R and RHadoop, as R is many people’s statistical tool of choice. Because this technique is broadly applicable to any situation involving resampling a large data set, we begin with a full general description of the problem and solution.

Formal problem statement for RHadoop

Our situation is as follows:

• We have N data points in our initial training set {xi}, where N is very large (106-109) and the data is distributed over a cluster.
• We want to train a set of M different models for an ensemble classifier, where M is anywhere from a handful to thousands.
• We want each model to be trained with K data points, where typically K << N. (For example, K may be 1–10% of.)

The number of training data points available to us, N, is fixed and generally outside of our control. However, K and M are both parameters that we can set and their product KM determines the total number of input vectors that will be consumed in the model fitting process. There are three cases to consider:

• KM < N, in which case we are not using the full amount of data available to us.
• KM = N, in which case we can exactly partition our data set to produce independent samples.
• KM > N, in which case we must resample some of our data with replacement.

The Poisson sampling method described below handles all three cases in the same framework. (However, note that for the case KM = N, it does not partition the data, but simply resamples it as well.)

(Note: The case where K = N corresponds exactly to bootstrapping the full initial data set, but this is often not desired for very large data sets. Nor is it practical from a computational perspective: performing a bootstrap of the full data set would require the generation of MN data points and M scans of an N-sized data set. However, in cases where this computation is desired, there exists an approximation called a “Bag of Little Bootstraps.”)

The goal

So our goal is to generate M data sets of size K from the original N data points where N can be very large and the data is sitting in a distributed environment. The two challenges we want to overcome are:

• Many resampling implementations perform M passes through the initial data set. which is highly undesirable in our case because the initial data set is so large.
• Sampling-with-replacement involves sampling from a multinomial distribution over the N input data points. However, sampling from a multinomial distribution requires message passing across the entire data set, so it is not possible to do so in a distributed environment in an embarrassingly parallel fashion (i.e., as a map-only MapReduce job).

Poisson-approximation resampling

Our solution to these issues is to approximate the multinomial sampling by sampling from a Poisson distribution for each input data point separately. For each input point xi, we sample M times from a Poisson(K / N) distribution to produce M values {mj}, one for each model j. For each data point xi and each model j, we emit the key-value pair *<j, xi>*a total of MJ times (where MJ can be zero). Because the sum of multiple Poisson variables is Poisson, the number of times a data point is emitted is distributed as Poisson(KM / N), and the size of each generated sample is distributed as Poisson(K), as desired. Because the Poisson sampling occurs for each input point independently, this sampling method can be parallelized in the map portion of a MapReduce job.

(Note that our approximation never guarantees that every single input data point is assigned to at least one of the models, but this is no worse than multinomial resampling of the full data set. However, in the case where KM = N, this is particularly bad in contrast to the alternative of partitioning the data, as partitioning will guarantee independent samples using all N training points, while resampling can only generate (hopefully) uncorrelated samples with a fraction of the data.)

Ultimately, each generated sample will have a size K on average, and so this method will approximate the exact multinomial sampling method with a single pass through the data in an embarrassingly parallel fashion, addressing both of the big data limitations described above. Because we are randomly sampling from the initial data set, and similarly to the “exact” method of multinomial sampling, some of the initial input vectors may never be chosen for any of the samples. We expect that approximately exp{–KM / N} of the initial data will be entirely missing from any of the samples (see figure below).

Amount of missed data as a function of KM / N. The value for KM = N is marked in gray.

Finally, the MapReduce shuffle distributes all the samples to the reducers and the model fitting or statistic computation is performed on the reduce side of the computation.

The algorithm for performing the sampling is presented below in pseudocode. Recall that there are three parameters —N, M, and K — where one is fixed; we choose to specify T = K / N as one of the parameters as it eliminates the need to determine the value of N in advance.

/# example sampling parameters

T = 0.1 # param 1: K / N; average fraction of input data in each model; 10%

M = 50 # param 2: number of models

def map(k, v): // for each input data point

for i in 1:M // for each model

m = Poisson(T) // num times curr point should appear in this sample 

if m > 0 

 for j in 1:m // emit current input point proper num of times 

    emit (i, v)

def reduce(k, v): 

fit model or calculate statistic with the sample in v

Note that even more significant performance enhancements can be achieved if it is possible to use a combiner, but this is highly statistic/model-dependent.

Example: Kaggle Data Set on Bulldozer Sale Prices
We will apply this method to test out the training of a random forest regression model on a Kaggle data set found here. The data set comprises ~400k training data points. Each data point represents a sale of a particular bulldozer at an auction, for which we have the sale price along with a set of other features about the sale and the bulldozer. (This data set is not especially large, but will illustrate our method nicely.) The goal will be to build a regression model using an ensemble method (specifically, a random forest) to predict the sale price of a bulldozer from the available features.

Could be yours for $141,999.99

The data are supplied as two tables: a transaction table that includes the sale price (target variable) and some other features, including a reference to a specific bulldozer; and a bulldozer table, that contains additional features for each bulldozer. As this post does not concern itself with data munging, we will assume that the data come pre-joined. But in a real-life situation, we’d incorporate the join as part of the workflow by, for example, processing it with a Hive query or a Pig script. Since in this case, the data are relatively small, we simply use some R commands. The code to prepare the data can be found here.

Quick note on R and RHadoop

As so much statistical work is performed in R, it is highly valuable to have an interface to use R over large data sets in a Hadoop cluster. This can be performed with RHadoop, which is developed with the support of Revolution Analytics. (Another option for R and Hadoop is the RHIPE project.)

One of the nice things about RHadoop is that R environments can be serialized and shuttled around, so there is never any reason to explicitly move any side data through Hadoop’s configuration or distributed cache. All environment variables are distributed around transparently to the user. Another nice property is that Hadoop is used quite transparently to the user, and the semantics allow for easily composing MapReduce jobs into pipelines by writing modular/reusable parts.

The only thing that might be unusual for the “traditional” Hadoop user (but natural to the R user) is that the mapper function should be written to be fully vectorized (i.e., keyval() should be called once per mapper as the last statement). This is to maximize the performance of the mapper (since R’s interpreted REPL is quite slow), but it means that mappers receive multiple input records at a time and everything the mappers emit must be grouped into a single object.

Finally, I did not find the RHadoop installation instructions (or the documentation in general) to be in a very mature state, so here are the commands I used to install RHadoop on my small cluster.

Fitting an ensemble of Random forests with poisson sampling on RHadoop

We implement our Poisson sampling strategy with RHadoop. We start by setting global values for our parameters:

frac.per.model <- 0.1 # 10% of input data to each sample on avg num.models <- 50

As mentioned previously, the mapper must deal with multiple input records at once, so there needs to be a bit of data wrangling before emitting the keys and values:

#MAPPER

poisson.subsample <- function(k, v) {

#parse data chunk into data frame 

#raw is basically a chunk of a csv file 

raw <- paste(v, sep="\n") 

#convert to data.frame using read.table() in parse.raw()

input <- parse.raw(raw)


#this function is used to generate a sample from

#the current block of data

generate.sample <- function(i) {

#generate N Poisson variables

draws <- rpois(n=nrow(input), lambda=frac.per.model)

#compute the index vector for the corresponding rows,

#weighted by the number of Poisson draws

indices <- rep((1:nrow(input))[draws > 0], draws[draws > 0])

#emit the rows; RHadoop takes care of replicating the key appropriately 

#and rbinding the data frames from different mappers together for the

#reducer 

keyval(rep(i, length(indices)), input[indices, ])

}

#here is where we generate the actual sampled data

raw.output <- lapply(1:num.models, generate.sample)


#and now we must reshape it into something RHadoop expects

output.keys <- do.call(c, lapply(raw.output, function(x) {x$key}))

output.vals <- do.call(rbind, lapply(raw.output, function(x) {x$val}))

keyval(output.keys, output.vals)

}

Because we are using R, the reducer can be incredibly simple: it takes the sample as an argument and simply feeds it to our model-fitting function, randomForest():

#REDUCE function 

fit.trees <- function(k, v) {

#rmr rbinds the emited values, so v is a dataframe 

#note that do.trace=T is used to produce output to stderr to keep

#the reduce task from timing out

rf <- randomForest(formula=model.formula,

    data=v,

    na.action=na.roughfix,

    ntree=10, do.trace=TRUE)

#rf is a list so wrap it in another list to ensure that only

#one object gets emitted. this is because keyval is vectorized

keyval(k, list(forest=rf))

 }

Keep in mind that in our case, we are actually fitting 10 trees per sample, but we could easily only fit a single tree per “forest”, and merge the results from each sample into a single real forest.

Note that the choice of predictors has specified in the variable model. formula. R’s random forest implementation does not support factors that have more than 32 levels, as the optimization problem grows too fast. To illustrate the Poisson sampling method, we chose to simply ignore those features, even though they probably contain useful information for regression. In a future blog post, we will address various ways that we can get around this limitation.

The MapReduce job itself is initiated like so:

mapreduce(input="/poisson/training.csv",

input.format="text", map=poisson.subsample,

reduce=fit.trees,

output="/poisson/output")

The resulting trees are dumped in HDFS at Poisson/output.

Finally, we can load the trees, merge them, and use them to classify new test points:

raw.forests <- from.dfs("/poisson/output")[["val"]]

forest <- do.call(combine, raw.forests)

Conclusion

Each of the 50 samples produced a random forest with 10 trees, so the final random forest is an ensemble of 500 trees, fitted in a distributed fashion over a Hadoop cluster. The full set of source files is available here.

Hopefully, you have now learned a scalable approach for training ensemble classifiers or bootstrapping in a parallel fashion by using a Poisson approximation to multinomial sampling.

Every cook knows how to avoid Type I Error: just remove the batteries. Let’s also learn how to reduce the chances of Type II errors. 

Why type I and type II errors matter

A/B testing is an essential component of large-scale online services today. So essential, that every online business worth mentioning has been doing it for the last 10 years.

A/B testing is also used in email marketing by all major online retailers. The Obama for America data science team received a lot of press coverage for leveraging data science, especially A/B testing during the presidential campaign.

Here is an interesting article on this topic along with a data science bootcamp that teaches a/b testing and statistical analysis.

If you have been involved in anything related to A/B testing (online experimentation) on UI, relevance or email marketing, chances are that you have heard of Type i and Type ii error. The usage of these terms is common but a good understanding of them is not.

I have seen illustrations as simple as this.

Examples of type I and type II errors

I intend to share two great examples I recently read that will help you remember this especially important concept in hypothesis testing.

Type I error: An alarm without a fire.

Type II error: A fire without an alarm.

Every cook knows how to avoid Type I Error – just remove the batteries. Unfortunately, this increases the incidences of Type II error.

Reducing the chances of Type II error would mean making the alarm hypersensitive, which in turn would increase the chances of Type I error.

Another way to remember this is by recalling the story of the Boy Who Cried Wolf.

Null hypothesis testing: There is no wolf.

Alternative hypothesis testing: There is a wolf.

Villagers believing the boy when there was no wolf (Reject the null hypothesis incorrectly): Type 1 Error. Villagers not believing the boy when there was a wolf (Rejecting alternative hypothesis incorrectly): Type 2 Error

Tailpiece

The purpose of the post is not to explain type 1 and type 2 error. If this is the first time you are hearing about these terms, here is the Wikipedia entry: Type I and Type II Error.

Ethics in research and A/B testing is essential. A/B testing might not be as simple and harmless as it looks. Learn how to take care of ethical concerns in A/B tests.

The ethical way to A/B testing

We have come a long way since the days of horrific human experiments during World Wars, the Stanford prison experiment, the Guatemalan STD Study, and many more where inhumane treatments were all in the name of science.

However, we still have much to learn, with incidents like the clinical trial disaster in France and Facebook’s emotional and psychological experiments of recent years violating the rights of persons and serving as a clear reminder to constantly keep our ethics in research sharply upfront.

As data scientists, we are always experimenting – not only with our models or formulas but also with the responses from our customers. A/B tests or randomized experiments may require human subjects, who are willing to undertake a trial or treatment such as seeing certain content when using a Web app, or undergoing a certain exercise regime.

Facebook example

What may initially seem like a harmless experiment, might cause harm or distress. For example, Facebook’s experiment of provoking negative emotions from some users and positive emotions from others could have grave consequences. If a user, who was experiencing emotional distress happened to have seen content that provoked negative feelings, it could spur on a tragic event such as physical harm.

Careful understanding of our experiments and our test subjects may prevent inappropriate testing required prior to implementing our research, or products and services. Consent is the best tool to assist data scientists working with data generated by people. Similarly, to guidelines for clinical trials, it’s informed consent specifically that is needed to avoid potential unintended consequences of experiments.

If an organization specializing in exercise science accepted participation from a person who has a high risk of heart failure and did not ask for a medical examination before experimenting, then the organization is potentially liable for the consequences.

Often a simple, harmless A/B test might not be as simple and harmless as it looks. So how do we ensure we are not putting our human subjects’ well-being and safety in danger when we conduct our research and experiments?

First steps in research

The first port of call is using informed user consent. This doesn’t mean pages and pages of legal jargon on sign-up or being vague in an email when reaching out for volunteers for your study. This could rather be a popup window or email that is clear on the purpose of the experiment and any warnings or potential risks the person needs to be aware of.

Depending on how intense the treatment is, a medical or psychological examination is a good idea to ensure that the participant can cope with the given treatment. Being unaware of people’s vulnerabilities can lead to unintended consequences. This can be avoided through clearer warnings or the next level up which may be online assessments or even expert examinations.

The next step in ensuring your A/B test or experiment runs smoothly and ethically is making sure you understand local and federal regulations around conducting research experiments on humans. In the US, these regulations have been outlined above. The regulations mainly look at:

• Informed consent, with a full explanation of any potential risks to the subject.
• Providing additional safeguards for vulnerable populations such as children, mentally disabled people, mentally ill people, economically disadvantaged people, pregnant women, and so on.
• Government-funded experiments need the approval of an Institutional Review Board or an independent ethics committee before conducting experiments.

During the A/B test or experiment, it’s also a good idea to regularly check in and see how your subjects are responding to the treatments, not only for scientific research but also to quickly solve any health or well-being issues.

This could be in the form of a short popup survey or email to check if the user is safe and well, or face-to-face consulting. Also, having an opt-out option allows the subject to take control if they feel their health or well-being is at risk. Having some people opt out might seem inconvenient for your study, but a serious or tragic incident as a result of a participant having to go through the full course of the treatment is a far worse outcome.

Observational studies might be a good alternative if the above steps are in no way feasible for your experiment. Observational studies are limited when making conclusions, and only real experiments allow you to make confident conclusions from the data. However, in some situations, it is not possible nor ethical to force treatments onto subjects.

For example, it’s not ethical to inject cancer cells into random subjects, but you can study cancer patients with the inherited attributes you are looking for to help with your research.

The ethical takeaway

It is understood that there can be some overhead in carefully preparing, setting up, and following ethical guidelines for an experiment or A/B test. However, the serious consequences of not doing it properly, as well as public distrust, will only lead to a reluctance to share data, hindering our ability to effectively do our work.

If you’re curious to learn more about A/B testing, watch the short video below.

What are Data Scientists? What job skills should they possess? Learn about the essential data scientist skills and their roles.

How do we distinguish a genuine data scientist from a dressed-up business analyst, BI, or other related roles?

Truth be told, the industry does not have a standard definition of a data scientist. You have probably heard jokes like “A data scientist is a data analyst living in Silicon Valley”. Just for fun, take a look at the below cartoon that demonstrates this.

Finding an “effective” data scientist is difficult. Finding people to play the role of a data scientist can be equally difficult. Note the use of “effective” here. I use this word to highlight the fact that there could be people who possess some of these data science skills yet may not be the best fit for a data science role. The irony is that even people looking to hire data scientists might not fully understand data science. There are still some job advertisements in the market that describe traditional data analyst and business analyst roles while labeling it a “Data Scientist” position.

Instead of giving a list of data science skills with bullet points, I will highlight the differences between some of the data-related roles.

Consider the following scenario:

Shop-Mart and Bulk-Mart are two competitors in the retail setting. Someone high up in the management chain asks this question: “How many Shop-Mart customers also go to Bulk-Mart?” Replace Shop-Mart and Bulk-Mart with Walmart, Target, Safeway, or any retail outlets that you know of. The question might be of interest to the management of one of these stores or even a third party. The third party could possibly be a market research or consumer behavior company interested in gathering actionable insights about consumer behavior.

How professionals in different data-related roles will approach the problem:

Traditional BI/Reporting professional:

The BI professional generates reports from structured data using SQL and some kind of reporting services (SSRS for example) and sends the data back to management. Management asks more questions based on the data that was sent, and the cycle continues. Insights about the data are most likely not included in the reports. A person in this role will be experienced mostly in database-related skills.

Data analyst:

In addition to doing what a BI professional does, a data analyst will also keep other factors like seasonality, segmentation, and visualization in mind. What if certain trends in shopping behavior are tied to seasonality?

What if the trends are different across gender, demographics, geography, or product category? A data analyst will slice and dice the data to understand and annotate the report. Aside from database skills, a data analyst will have an understanding of some of the common visualization tools.

Business analyst:

A business analyst possesses the skills of a BI professional and a data analyst, plus they have domain knowledge and an understanding of the business. A business analyst may also have some basic skills in forecasting.

Data mining or big data engineer:

A data miner does the job of the data analyst, possibly from unstructured data if needed, plus possesses MapReduce and other big data skills. An understanding of common issues in running jobs on large-scale data and debugging of MapReduce jobs is needed.

Statistician (a traditional one):

A statistician pulls data from a database or obtains it from any of the roles mentioned above and performs statistical analysis. This person ensures the quality of data and correctness of the conclusions by using standard practices like choosing the right sample size, confidence level, level of significance, type of test, and so on.

In the past, statisticians did not traditionally come from a computer science background, needed for writing code to implement statistical models. The situation has changed, Stat students now graduating with strong programming skills and decent foundation skills in CS. This enables them to perform the tasks that previous statisticians were not trained for traditionally.

Program/Project manager:

The program or project manager looks at all the data provided by the professionals mentioned so far, aligns these findings with the business, and influences the leadership to take appropriate action. This person possesses communication and presentation skills that can influence others without authority.

Ironically a PM is influencing business decisions using the data and insights provided by others. If the person does not have a knack for understanding data, chances are that they will not be able to influence others to make the best decisions.

Now, putting it all together

The rise of online services has brought a paradigm shift in the software development life cycle and business iteration over successive features and products. Having a different data puller, analyst, statistician, and project manager is just not possible anymore. Now the mantra is: ship, experiment, and learn; adapt; ship, experiment, and learn. This situation has resulted in the birth of a new role: a data scientist.

The mentioned qualities make up the needed data scientist skills. In addition to the skills mentioned above, a data scientist should have rapid prototyping and programming, machine learning, visualization, and hacking skills.

Domain knowledge and soft skills are equally important as technical skills

The importance of domain knowledge and soft skills, like communication and influencing without authority, are severely underestimated both by hiring managers and aspiring data scientists. Insights without domain knowledge can potentially mislead the consumers of these insights. Correct insights without the ability to influence decision-making are just as bad as having no insights.

All of what I have said above is based on my own tenure as a data scientist at a major search engine and later with the advertising platform within the same company. I learned that sometimes people asking the question may not understand what they want to know. This sounds preposterous yet it happens way too often.

Very often a bozo will start digging into something that is not related to the issue at hand just to prove that he/she is relevant. A data scientist encounters such HIPPOs (Highly Paid Person’s Opinions) that are somewhat unrelated to the problem and are very often a big distraction from the problem at hand.

Data scientist skills must include the ability to manage situations such as people asking irrelevant, distracting questions that are outside the scope of the task at hand. This is hard, especially in situations where the person asking the question is several levels up the corporate ladder and is known to have an ego. It is a data scientist’s responsibility to manage up and around while presenting and communicating insights.

Suggested skills a data science expert should possess

Curiosity about data and passion for the domain

If you are not passionate about the domain or business, and if you are not curious about data, then it is unlikely that you will succeed in a data scientist role. If you are working with an online retailer, your data scientist skills should be hungry to crunch and munch from the smorgasbord (of data, of course) to know more. If your curiosity does not keep you awake, no skill in the world can help you succeed.

Soft skills:

Communication and influencing without authority are necessary skills. Understand the minimum action that has the maximum impact. Too many findings are as bad as no findings at all.  The ability to scoop information out of partners and customers, even from the unwilling ones, is extremely important. The data you are looking for may not be sitting in one single place. You may have to beg, borrow, steal, and do whatever it takes to get the data.

Being a good storyteller is also something that helps. Sometimes the insights obtained from data are counter-intuitive. If you’re not a good storyteller, it will be difficult to convince your audience.

Math/Theory

Machine Learning algorithms, statistics, and probability 101 are fundamental to data science. This includes understanding probability distributions, linear regression, statistical inference, hypothesis testing, and confidence intervals. Learning optimization, such as gradient descent, would be the icing on the cake.

Computer science/programming

You should know at least one scripting language (I prefer Python), or a statistical tool such as R. There are plenty of resources to get started. Data Science Dojo provides numerous, free tutorials on getting started with Python and R. You can also learn the basics of programming from sites like CodeAcademy and LearnPython.

It is necessary to possess decent algorithms and DS skills in order to write code that can analyze a lot of data efficiently. You may not be a production code developer, but you should be able to write decent code.

Database management and SQL skills are also helpful, as this is where you will be fetching your data to build models. It also doesn’t hurt to understand Microsoft Excel or another spreadsheet software.

Big data and distributed systems

You need to understand basic MapReduce concepts, the Hadoop and Hadoop file systems, and at least one language like Hive/Pig. Some companies have their own proprietary implementations of these languages.
Knowledge of tools like Mahout and any of the XaaS, like Azure and AWS, would be helpful. Once again, big companies have their own XaaS, so you may be working on variants of any of these.

Visualization

Possess the ability to create simple yet elegant and meaningful visualizations. Personally, R packages like ggplot, lattice, and others have helped me in most cases, but there are other packages that you can use. In some cases, you might want to use D3.

A visualization of data scientist skills:

Where are data scientists in the big data pipeline?

Below is a visualization of the big data pipeline, the associated technologies, and the regions of operation. In general, the depiction of where the data scientist belongs in this pipeline is largely correct, but there is one caveat. Data scientist skills must include being comfortable about diving into the “Collect” and “Store” territories if needed. Usually, data scientists are working on transformed data and beyond. However, in scenarios where the business cannot afford to wait for the transformation process to finish, data scientist skills enable them to work on raw data to gather insights.

Are you ready to become a data scientist? Are you interested in possessing the data scientist skills? Learn the foundation of data science and start implementing your models in our time convenient, 5-day bootcamp. Check it out here!