How would you create a taxonomy to identify key customer trends in unstructured

data?

Tweet: Data Science Interview questions #1 - How would you create a taxonomy

to identify key customer trends in unstructured data? - http://ctt.ec/sdqZ0+

The best way to approach this question is to mention that it is good to check

with the business owner and understand their objectives before categorizing the

data. Having done this, it is always good to follow an iterative approach by

pulling new data samples and improving the model accordingly by validating it

for accuracy by soliciting feedback from the stakeholders of the business. This

helps ensure that your model is producing actionable results and improving over

the time.

2) Python or R – Which one would you prefer for text

analytics?

The best possible answer for this would be Python because it has Pandas library

that provides easy to use data structures and high performance data analysis

tools.

3) Which technique is used to predict categorical responses?

Classification technique is used widely in mining for classifying data sets.

CLICK HERE to get the data scientist salary report delivered to your inbox!

4) What is logistic regression? Or State an example when you

have used logistic regression recently.

Logistic Regression often referred as logit model is a technique to predict the

binary outcome from a linear combination of predictor variables. For example,

if you want to predict whether a particular political leader will win the election

or not. In this case, the outcome of prediction is binary i.e. 0 or 1 (Win/Lose).

The predictor variables here would be the amount of money spent for election

campaigning of a particular candidate, the amount of time spent in campaigning,

etc.

5) What are Recommender Systems?

A subclass of information filtering systems that are meant to predict the

preferences or ratings that a user would give to a product. Recommender systems

are widely used in movies, news, research articles, products, social tags, music,

etc.

6) Why data cleaning plays a vital role in analysis?

Cleaning data from multiple sources to transform it into a format that data

analysts or data scientists can work with is a cumbersome process because - as

the number of data sources increases, the time take to clean the data increases

exponentially due to the number of sources and the volume of data generated in

these sources. It might take up to 80% of the time for just cleaning data making

it a critical part of analysis task.

7) Differentiate between univariate, bivariate and

multivariate analysis.

These are descriptive statistical analysis techniques which can be

differentiated based on the number of variables involved at a given point of

time. For example, the pie charts of sales based on territory involve only one

variable and can be referred to as univariate analysis.

If the analysis attempts to understand the difference between 2 variables at

time as in a scatterplot, then it is referred to as bivariate analysis. For

example, analysing the volume of sale and a spending can be considered as an

example of bivariate analysis.

Analysis that deals with the study of more than two variables to understand the

effect of variables on the responses is referred to as multivariate analysis.

8) What do you understand by the term Normal Distribution?

Data is usually distributed in different ways with a bias to the left or to the

right or it can all be jumbled up. However, there are chances that data is

distributed around a central value without any bias to the left or right and

reaches normal distribution in the form of a bell shaped curve. The random

variables are distributed in the form of an symmetrical bell shaped curve.

Image Credit : mathisfun.com

9) What is Linear Regression?

Linear regression is a statistical technique where the score of a variable Y

is predicted from the score of a second variable X. X is referred to as the

predictor variable and Y as the criterion variable.

10) What is Interpolation and Extrapolation?

Estimating a value from 2 known values from a list of values is Interpolation.

Extrapolation is approximating a value by extending a known set of values or

facts.

11) What is power analysis?

An experimental design technique for determining the effect of a given sample

size.

12) What is K-means? How can you select K for K-means?

13) What is Collaborative filtering?

The process of filtering used by most of the recommender systems to find patterns

or information by collaborating viewpoints, various data sources and multiple

agents.

14) What is the difference between Cluster and Systematic

Sampling?

Cluster sampling is a technique used when it becomes difficult to study the target

population spread across a wide area and simple random sampling cannot be applied.

Cluster Sample is a probability sample where each sampling unit is a collection,

or cluster of elements. Systematic sampling is a statistical technique where

elements are selected from an ordered sampling frame. In systematic sampling,

the list is progressed in a circular manner so once you reach the end of the

list,it is progressed from the top again. The best example for systematic

sampling is equal probability method.

15) Are expected value and mean value different?

They are not different but the terms are used in different contexts. Mean is

generally referred when talking about a probability distribution or sample

population whereas expected value is generally referred in a random variable

context.

For Sampling Data

Mean value is the only value that comes from the sampling data.

Expected Value is the mean of all the means i.e. the value that is built from

multiple samples. Expected value is the population mean.

For Distributions

Mean value and Expected value are same irrespective of the distribution, under

the condition that the distribution is in the same population.

16) What does P-value signify about the statistical data?

P-value is used to determine the significance of results after a hypothesis test

in statistics. P-value helps the readers to draw conclusions and is always

between 0 and 1.

• P- Value > 0.05 denotes weak evidence against the null

hypothesis which means the null hypothesis cannot be rejected.

• P-value <= 0.05 denotes strong evidence against the null

hypothesis which means the null hypothesis can be rejected.

• P-value=0.05is the marginal value indicating it is

possible to go either way.

17) Do gradient descent methods always converge to same point?

No, they do not because in some cases it reaches a local minima or a local optima

point. You don’t reach the global optima point. It depends on the data and

starting conditions

18) What are categorical variables?

19) A test has a true positive rate of 100% and false positive rate

of 5%. There is a population with a 1/1000 rate of having the condition the

test identifies. Considering a positive test, what is the probability of having

that condition?

Let’s suppose you are being tested for a disease, if you have the illness the

test will end up saying you have the illness. However, if you don’t have the

illness- 5% of the times the test will end up saying you have the illness and

95% of the times the test will give accurate result that you don’t have the

illness. Thus there is a 5% error in case you do not have the illness.

Out of 1000 people, 1 person who has the disease will get true positive result.

Out of the remaining 999 people, 5% will also get true positive result.

Close to 50 people will get a true positive result for the disease.
This means that out of 1000 people, 51 people will be tested positive for the

disease even though only one person has the illness. There is only a 2%

probability of you having the disease even if your reports say that you have

the disease.

20) How you can make data normal using Box-Cox transformation?

21) What is the difference between Supervised Learning an

Unsupervised Learning?

If an algorithm learns something from the training data so that the knowledge

can be applied to the test data, then it is referred to as Supervised Learning.

Classification is an example for Supervised Learning. If the algorithm does not

learn anything beforehand because there is no response variable or any training

data, then it is referred to as unsupervised learning. Clustering is an example

for unsupervised learning.

22) Explain the use of Combinatorics in data science.

23) Why is vectorization considered a powerful method for optimizing numerical

code?

24) What is the goal of A/B Testing?

It is a statistical hypothesis testing for randomized experiment with two

variables A and B. The goal of A/B Testing is to identify any changes to the

web page to maximize or increase the outcome of an interest. An example for this

could be identifying the click through rate for a banner ad.

25) What is an Eigenvalue and Eigenvector?

Eigenvectors are used for understanding linear transformations. In data analysis,

we usually calculate the eigenvectors for a correlation or covariance matrix.

Eigenvectors are the directions along which a particular linear transformation

acts by flipping, compressing or stretching. Eigenvalue can be referred to as

the strength of the transformation in the direction of eigenvector or the factor

by which the compression occurs.

26) What is Gradient Descent?

27) How can outlier values be treated?

Outlier values can be identified by using univariate or any other graphical

analysis method. If the number of outlier values is few then they can be assessed

individually but for large number of outliers the values can be substituted with

either the 99th or the 1st percentile values. All extreme values are not outlier

values.The most common ways to treat outlier values –

1) To change the value and bring in within a range

2) To just remove the value.

28) How can you assess a good logistic model?

There are various methods to assess the results of a logistic regression

analysis-

• Using Classification Matrix to look at the true negatives

and false positives.

• Concordance that helps identify the ability of the

logistic model to differentiate between the event happening and not happening.

• Lift helps assess the logistic model by comparing it with

random selection.

29) What are various steps involved in an analytics project?

• Understand the business problem

• Explore the data and become familiar with it.

• Prepare the data for modelling by detecting outliers,

treating missing values, transforming variables, etc.

 • After data preparation, start running the model, analyse

the result and tweak the approach. This is an iterative step till the best

possible outcome is achieved.

• Validate the model using a new data set.

• Start implementing the model and track the result to

analyse the performance of the model over the period of time.

30) How can you iterate over a list and also retrieve element indices at the

same time?

This can be done using the enumerate function which takes every element in a

sequence just like in a list and adds its location just before it.

31) During analysis, how do you treat missing values?

The extent of the missing values is identified after identifying the variables

with missing values. If any patterns are identified the analyst has to

concentrate on them as it could lead to interesting and meaningful business

insights. If there are no patterns identified, then the missing values can be

substituted with mean or median values (imputation) or they can simply be

ignored.There are various factors to be considered when answering this question-

 Understand the problem statement, understand the data and then give the

answer.Assigning a default value which can be mean, minimum or maximum value.

Getting into the data is important.

 If it is a categorical variable, the default value is assigned. The

missing value is assigned a default value.

 If you have a distribution of data coming, for normal distribution give

the mean value.

 Should we even treat missing values is another important point to consider?

If 80% of the values for a variable are missing then you can answer that you

would be dropping the variable instead of treating the missing values.

32) Explain about the box cox transformation in regression models.

33) Can you use machine learning for time series analysis?

Yes, it can be used but it depends on the applications.

34) Write a function that takes in two sorted lists and outputs

a sorted list that is their union.

First solution which will come to your mind is to merge two lists and short them

afterwards

Python code-

def return_union(list_a, list_b):

return sorted(list_a + list_b)

R code-

return_union <- function(list_a, list_b)

list_c<-list(c(unlist(list_a),unlist(list_b)))

return(list(list_c[[1]][order(list_c[[1]])]))

Generally, the tricky part of the question is not to use any sorting or ordering

function. In that case you will have to write your own logic to answer the question

and impress your interviewer.

Python code-

def return_union(list_a, list_b):

len1 = len(list_a)

len2 = len(list_b)

final_sorted_list = []
 j = 0

k = 0

for i in range(len1+len2):

if k == len1:

final_sorted_list.extend(list_b[j:])

break

elif j == len2:

final_sorted_list.extend(list_a[k:])

break

elif list_a[k] < list_b[j]:

final_sorted_list.append(list_a[k])

k += 1

else:

final_sorted_list.append(list_b[j])

j += 1

return final_sorted_list

Similar function can be returned in R as well by following the similar steps.

return_union <- function(list_a,list_b)

#Initializing length variables

len_a <- length(list_a)

len_b <- length(list_b)

len <- len_a + len_b

#initializing counter variables

j=1

k=1

#Creating an empty list which has length equal to sum of both the lists
list_c <- list(rep(NA,len))

#Here goes our for loop

for(i in 1:len)

if(j>len_a)

list_c[i:len] <- list_b[k:len_b]

break

else if(k>len_b)

list_c[i:len] <- list_a[j:len_a]

break

else if(list_a[[j]] <= list_b[[k]])

list_c[[i]] <- list_a[[j]]

j <- j+1

else if(list_a[[j]] > list_b[[k]])

list_c[[i]] <- list_b[[k]]

k <- k+1

return(list(unlist(list_c)))

35) What is the difference between Bayesian Inference and Maximum

Likelihood Estimation (MLE)?

36) What is Regularization and what kind of problems does

regularization solve?

37) What is multicollinearity and how you can overcome it?

38) What is the curse of dimensionality?

39) How do you decide whether your linear regression model fits

the data?

40) What is the difference between squared error and absolute

error?

41) What is Machine Learning?

The simplest way to answer this question is – we give the data and equation

to the machine. Ask the machine to look at the data and identify the coefficient

values in an equation.

For example for the linear regression y=mx+c, we give the data for the variable

x, y and the machine learns about the values of m and c from the data.

42) How are confidence intervals constructed and how will you interpret them?

43) How will you explain logistic regression to an economist, physican

scientist and biologist?

44) How can you overcome Overfitting?

45) Differentiate between wide and tall data formats?

46) Is Naïve Bayes bad? If yes, under what aspects.

47) How would you develop a model to identify plagiarism?

48) How will you define the number of clusters in a clustering algorithm?

Though the Clustering Algorithm is not specified, this question will mostly be

asked in reference to K-Means clustering where “K” defines the number of

clusters. The objective of clustering is to group similar entities in a way

that the entities within a group are similar to each other but the groups are

different from each other.

For example, the following image shows three different groups.

Within Sum of squares is generally used to explain the homogeneity within a

cluster. If you plot WSS for a range of number of clusters, you will get the

plot shown below. The Graph is generally known as Elbow Curve.

Red circled point in above graph i.e. Number of Cluster =6 is the point after

which you don’t see any decrement in WSS. This point is known as bending point

and taken as K in K – Means.

This is the widely used approach but few data scientists also use Hierarchical

clustering first to create dendograms and identify the distinct groups from

there.

49) Is it better to have too many false negatives or too many false positives?

50) Is it possible to perform logistic regression with Microsoft Excel?

It is possible to perform logistic regression with Microsoft Excel. There are

two ways to do it using Excel.

a) One is to use Add-ins provided by many websites which we can use.

b) Second is to use fundamentals of logistic regression and use Excel’

s computational power to build a logistic regression

But when this question is being asked in an interview, interviewer is not looking

for a name of Add-ins rather a method using the base excel functionalities.

Let’s use a sample data to learn about logistic regression using Excel. (Example

assumes that you are familiar with basic concepts of logistic regression)
Data shown above consists of three variables where X1 and X2 are independent

variables and Y is a class variable. We have kept only 2 categories for our purpose

of binary logistic regression classifier.

Next we have to create a logit function using independent variables, i.e.

Logit = L = β0 + β1*X1 + β2*X2

51) What do you understand by Fuzzy merging ? Which language will you use

to handle it?

52) What is the difference between skewed and uniform distribution?

53) You created a predictive model of a quantitative outcome variable using

multiple regressions. What are the steps you would follow to validate the model?

Since the question asked, is about post model building exercise, we will assume

that you have already tested for null hypothesis, multi collinearity and Standard

error of coefficients.

Once you have built the model, you should check for following –

· Global F-test to see the significance of group of independent

variables on dependent variable

· R^2

· Adjusted R^2

· RMSE, MAPE

In addition to above mentioned quantitative metrics you should also check for-

· Residual plot

· Assumptions of linear regression

54) What do you understand by Hypothesis in the content of Machine Learning?

55) What do you understand by Recall and Precision?

56) How will you find the right K for K-means?

57) Why L1 regularizations causes parameter sparsity whereas L2 regularization

does not?

Regularizations in statistics or in the field of machine learning is used to

include some extra information in order to solve a problem in a better way. L1

& L2 regularizations are generally used to add constraints to optimization

problems.

In the example shown above H0 is a hypothesis. If you observe, in L1 there

is a high likelihood to hit the corners as solutions while in L2, it doesn’

t. So in L1 variables are penalized more as compared to L2 which results into

sparsity.

In other words, errors are squared in L2, so model sees higher error and tries

to minimize that squared error.

58) How can you deal with different types of seasonality in time series

modelling?

Seasonality in time series occurs when time series shows a repeated pattern over

time. E.g., stationary sales decreases during holiday season, air conditioner

sales increases during the summers etc. are few examples of seasonality in a

time series.

Seasonality makes your time series non-stationary because average value of the

variables at different time periods. Differentiating a time series is generally

known as the best method of removing seasonality from a time series. Seasonal
differencing can be defined as a numerical difference between a particular

value and a value with a periodic lag (i.e. 12, if monthly seasonality is present)

59) In experimental design, is it necessary to do randomization? If yes, why?

60) What do you understand by conjugate-prior with respect to Naïve Bayes?

61) Can you cite some examples where a false positive is important than a false

negative?

Before we start, let us understand what are false positives and what are false

negatives.

False Positives are the cases where you wrongly classified a non-event as an

event a.k.a Type I error.

And, False Negatives are the cases where you wrongly classify events as

non-events, a.k.a Type II error.

In medical field, assume you have to give chemo therapy to patients. Your lab

tests patients for certain vital information and based on those results they

decide to give radiation therapy to a patient.

Assume a patient comes to that hospital and he is tested positive for cancer

(But he doesn’t have cancer) based on lab prediction. What will happen to him?

(Assuming Sensitivity is 1)

One more example might come from marketing. Let’s say an ecommerce company

decided to give $1000 Gift voucher to the customers whom they assume to purchase

at least $5000 worth of items. They send free voucher mail directly to 100

customers without any minimum purchase condition because they assume to make

at least 20% profit on sold items above 5K.

Now what if they have sent it to false positive cases?

62) Can you cite some examples where a false negative important than a false

positive?

Assume there is an airport ‘A’ which has received high security threats and

based on certain characteristics they identify whether a particular passenger

can be a threat or not. Due to shortage of staff they decided to scan passenger

being predicted as risk positives by their predictive model.

What will happen if a true threat customer is being flagged as non-threat by

airport model?

Another example can be judicial system. What if Jury or judge decide to make

a criminal go free?

What if you rejected to marry a very good person based on your predictive model

and you happen to meet him/her after few years and realize that you had a false

negative?

63) Can you cite some examples where both false positive and false negatives

are equally important?

In the banking industry giving loans is the primary source of making money but

at the same time if your repayment rate is not good you will not make any profit,

rather you will risk huge losses.

Banks don’t want to lose good customers and at the same point of time they don’

t want to acquire bad customers. In this scenario both the false positives and

false negatives become very important to measure.

These days we hear many cases of players using steroids during sport competitions

Every player has to go through a steroid test before the game starts. A false

positive can ruin the career of a Great sportsman and a false negative can make

the game unfair.

64) Can you explain the difference between a Test Set and a Validation Set?
 Validation set can be considered as a part of the training set as it is used

for parameter selection and to avoid Overfitting of the model being built. On

the other hand, test set is used for testing or evaluating the performance of

a trained machine leaning model.

In simple terms ,the differences can be summarized as-

 Training Set is to fit the parameters i.e. weights.

 Test Set is to assess the performance of the model i.e. evaluating the

predictive power and generalization.

 Validation set is to tune the parameters.

65) What makes a dataset gold standard?

66) What do you understand by statistical power of sensitivity and how do you

calculate it?

Sensitivity is commonly used to validate the accuracy of a classifier (Logistic,

SVM, RF etc.). Sensitivity is nothing but “Predicted TRUE events/ Total events”.

True events here are the events which were true and model also predicted them

as true.

Calculation of seasonality is pretty straight forward-

Seasonality = True Positives /Positives in Actual Dependent Variable

Where, True positives are Positive events which are correctly classified as

Positives.

67) What is the importance of having a selection bias?

68) Give some situations where you will use an SVM over a RandomForest Machine

Learning algorithm and vice-versa.

SVM and Random Forest are both used in classification problems.

a) If you are sure that your data is outlier free and clean then go

for SVM. It is the opposite - if your data might contain outliers then Random

forest would be the best choice

b) Generally, SVM consumes more computational power than Random

Forest, so if you are constrained with memory go for Random Forest machine

learning algorithm.

c) Random Forest gives you a very good idea of variable importance in your

data, so if you want to have variable importance then choose Random Forest machine

learning algorithm.

d) Random Forest machine learning algorithms are preferred for

multiclass problems.

e) SVM is preferred in multi-dimensional problem set - like text

classification

but as a good data scientist, you should experiment with both of them and test

for accuracy or rather you can use ensemble of many Machine Learning techniques.

69) What do you understand by feature vectors?

70) How do data management procedures like missing data handling make selection

bias worse?

Missing value treatment is one of the primary tasks which a data scientist is

supposed to do before starting data analysis. There are multiple methods for

missing value treatment. If not done properly, it could potentially result into

selection bias. Let see few missing value treatment examples and their impact

on selection-

Complete Case Treatment: Complete case treatment is when you remove entire row

in data even if one value is missing. You could achieve a selection bias if your

values are not missing at random and they have some pattern. Assume you are
conducting a survey and few people didn’t specify their gender. Would you remove

all those people? Can’t it tell a different story?

Available case analysis: Let say you are trying to calculate correlation matrix

for data so you might remove the missing values from variables which are needed

for that particular correlation coefficient. In this case your values will not

be fully correct as they are coming from population sets.

Mean Substitution: In this method missing values are replaced with mean of

other available values.This might make your distribution biased e.g., standard

deviation, correlation and regression are mostly dependent on the mean value

of variables.

Hence, various data management procedures might include selection bias in your

data if not chosen correctly.

71) What are the advantages and disadvantages of using regularization methods

like Ridge Regression?

72) What do you understand by long and wide data formats?

73) What do you understand by outliers and inliers? What would you do if you

find them in your dataset?

74) Write a program in Python which takes input as the diameter of a coin and

weight of the coin and produces output as the money value of the coin.

75) What are the basic assumptions to be made for linear regression?

Normality of error distribution, statistical independence of errors, linearity

and additivity.

76) Can you write the formula to calculat R-square?

R-Square can be calculated using the below formular -

1 - (Residual Sum of Squares/ Total Sum of Squares)

77) What is the advantage of performing dimensionality reduction before

fitting an SVM?

Support Vector Machine Learning Algorithm performs better in the reduced space.

It is beneficial to perform dimensionality reduction before fitting an SVM if

the number of features is large when compared to the number of observations.

78) How will you assess the statistical significance of an insight whether it

is a real insight or just by chance?

Statistical importance of an insight can be accessed using Hypothesis Testing.

Learn Data Science in Python and R Programming to nail data science

interviews at top tech companies!

Data Science Puzzles-Brain Storming/ Puzzle based Data Science Interview

Questions asked in Data Scientist Job Interviews

1) How many Piano Tuners are there in Chicago?

To solve this kind of a problem, we need to know –

Can you tell if the equation given below is linear or not ?

Emp_sal= 2000+2.5(emp_age)2

Yes it is a linear equation as the coefficients are linear.

What will be the output of the following R programming code ?

var2<- c("I","Love,"DeZyre")

var2

It will give an error.

How many Pianos are there in Chicago?

How often would a Piano require tuning?

How much time does it take for each tuning?

We need to build these estimates to solve this kind of a problem. Suppose, let’

s assume Chicago has close to 10 million people and on an average there are 2

people in a house. For every 20 households there is 1 Piano. Now the question

how many pianos are there can be answered. 1 in 20 households has a piano, so

approximately 250,000 pianos are there in Chicago.

Now the next question is-“How often would a Piano require tuning? There is no

exact answer to this question. It could be once a year or twice a year. You need

to approach this question as the interviewer is trying to test your knowledge

on whether you take this into consideration or not. Let’s suppose each piano

requires tuning once a year so on the whole 250,000 piano tunings are required.

Let’s suppose that a piano tuner works for 50 weeks in a year considering a

5 day week. Thus a piano tuner works for 250 days in a year. Let’s suppose tuning

a piano takes 2 hours then in an 8 hour workday the piano tuner would be able

to tune only 4 pianos. Considering this rate, a piano tuner can tune 1000 pianos

a year.

Thus, 250 piano tuners are required in Chicago considering the above estimates.

2) There is a race track with five lanes. There are 25 horses of which you want

to find out the three fastest horses. What is the minimal number of races needed

to identify the 3 fastest horses of those 25?

Divide the 25 horses into 5 groups where each group contains 5 horses. Race

between all the 5 groups (5 races) will determine the winners of each group.

A race between all the winners will determine the winner of the winners and must

be the fastest horse. A final race between the 2nd and 3rd place from the

winners group along with the 1st and 2nd place of thee second place group along

with the third place horse will determine the second and third fastest horse

from the group of 25.

3) Estimate the number of french fries sold by McDonald's everyday.

4) How many times in a day does a clock’s hand overlap?

 5) You have two beakers. The first beaker contains 4 litre of water and the

second one contains 5 litres of water.How can you our exactly 7 litres of water

into a bucket?

6) A coin is flipped 1000 times and 560 times heads show up. Do you think the

coin is biased?

7) Estimate the number of tennis balls that can fit into a plane.

8) How many haircuts do you think happen in US every year?

9) In a city where residents prefer only boys, every family in the city

continues to give birth to children until a boy is born. If a girl is born,

they plan for another child. If a boy is born, they stop. Find out the proportion

of boys to girls in the city.

Probability Interview Questions for Data Science

1. There are two companies manufacturing electronic chip. Company A is

manufactures defective chips with a probability of 20% and good quality chips

with a probability of 80%. Company B manufactures defective chips with a

probability of 80% and good chips with a probability of 20%.If you get just one

electronic chip, what is the probability that it is a good chip?

2. Suppose that you now get a pack of 2 electronic chips coming from the

same company either A or B. When you test the first electronic chip it appears

to be good. What is the probability that the second electronic chip you received

is also good?

3. A dating site allows users to select 6 out of 25 adjectives to describe

their likes and preferences. A match is said to be found between two users on

the website if the match on atleast 5 adjectives. If Steve and On a dating site,

users can select 5 out of 24 adjectives to describe themselves. A match is

declared between two users if they match on at least 4 adjectives. If Brad and

Angelina randomly pick adjectives, what is the probability that they will form

a match?
4. A coin is tossed 10 times and the results are 2 tails and 8 heads. How

will you analyse whether the coin is fair or not? What is the p-value for the

same?

5. Continuation to the above question, if each coin is tossed 10 times (100

tosses are made in total). Will you modify your approach to the test the fairness

of the coin or continue with the same?

6. An ant is placed on an infinitely long twig. The ant can move one step

backward or one step forward with same probability during discrete time steps.

Find out the probability with which the ant will return to the starting point.

Statistics Interview Questions for Data Science

1. Explain the central limit theorem.

2. What is the relevance of central limit theorem to a class of freshmen in the

social sciences who hardly have any knowledge about statistics?

3. Given a dataset, show me how Euclidean Distance works in three dimensions.

4. How will you prevent overfitting when creating a statistical model ?

Frequently Asked Open Ended Machine Learning Interview Questions for Data
Scientists

1. Which is your favourite machine learning algorithm and why?

2. In which libraries for Data Science in Python and R, does your strength

lie?

3. What kind of data is important for specific business requirements and

how, as a data scientist will you go about collecting that data?

4. Tell us about the biggest data set you have processed till date and for

what kind of analysis.

5. Which data scientists you admire the most and why?

6. Suppose you are given a data set, what will you do with it to find out

if it suits the business needs of your project or not.

7. What were the business outcomes or decisions for the projects you worked

on?

8. What unique skills you think can you add on to our data science team?

9. Which are your favorite data science startups?

10. Why do you want to pursue a career in data science?

11. What have you done to upgrade your skills in analytics?

12. What has been the most useful business insight or development you have

found?

13. How will you explain an A/B test to an engineer who does not know

statistics?

14. When does parallelism helps your algorithms run faster and when does it

make them run slower?

15. How can you ensure that you don’t analyse something that ends up producing

meaningless results?

16. How would you explain to the senior management in your organization as

to why a particular data set is important?

17. Is more data always better?

18. What are your favourite imputation techniques to handle missing data?

19. What are your favorite data visualization tools?

20. Explain the life cycle of a data science project.

These are some more questions with no answers try finding it out

1. What is the biggest data set that you processed, and how did you process
it, what were the results?
2. Tell me two success stories about your analytic or computer science
projects? How was lift (or success) measured?
3. What is: lift, KPI, robustness, model fitting, design of experiments,
80/20 rule?
4. What is: collaborative filtering, n-grams, map reduce, cosine distance?
5. How to optimize a web crawler to run much faster, extract better
information, and better summarize data to produce cleaner databases?
6. How would you come up with a solution to identify plagiarism?
7. How to detect individual paid accounts shared by multiple users?
8. Should click data be handled in real time? Why? In which contexts?
9. What is better: good data or good models? And how do you define "good"?
Is there a universal good model? Are there any models that are definitely
not so good?
10. What is probabilistic merging (AKA fuzzy merging)? Is it easier to handle
with SQL or other languages? Which languages would you choose for
semi-structured text data reconciliation?
11. How do you handle missing data? What imputation techniques do you
recommend?
12. What is your favorite programming language / vendor? why?
13. Tell me 3 things positive and 3 things negative about your favorite
statistical software.
14. Compare SAS, R, Python, Perl
15. What is the curse of big data?
16. Have you been involved in database design and data modeling?
17. Have you been involved in dashboard creation and metric selection? What
do you think about Birt?
18. What features of Teradata do you like?
19. You are about to send one million email (marketing campaign). How do you
optimze delivery? How do you optimize response? Can you optimize both
separately? (answer: not really)
20. Toad or Brio or any other similar clients are quite inefficient to query
Oracle databases. Why? How would you do to increase speed by a factor 10,
and be able to handle far bigger outputs?
21. How would you turn unstructured data into structured data? Is it really
necessary? Is it OK to store data as flat text files rather than in an
SQL-powered RDBMS?
22. What are hash table collisions? How is it avoided? How frequently does
it happen?
23. How to make sure a mapreduce application has good load balance? What is
load balance?
24. Examples where mapreduce does not work? Examples where it works very well?
What are the security issues involved with the cloud? What do you think
of EMC's solution offering an hybrid approach - both internal and external
cloud - to mitigate the risks and offer other advantages (which ones)?
25. Is it better to have 100 small hash tables or one big hash table, in memory,
in terms of access speed (assuming both fit within RAM)? What do you think
about in-database analytics?
26. Why is naive Bayes so bad? How would you improve a spam detection algorithm
that uses naive Bayes?
27. Have you been working with white lists? Positive rules? (In the context
of fraud or spam detection)
28. What is star schema? Lookup tables?
29. Can you perform logistic regression with Excel? (yes) How? (use linest
on log-transformed data)? Would the result be good? (Excel has numerical
issues, but it's very interactive)
30. Have you optimized code or algorithms for speed: in SQL, Perl, C++, Python
etc. How, and by how much?
31. Is it better to spend 5 days developing a 90% accurate solution, or 10
days for 100% accuracy? Depends on the context?
32. Define: quality assurance, six sigma, design of experiments. Give
examples of good and bad designs of experiments.
33. What are the drawbacks of general linear model? Are you familiar with
alternatives (Lasso, ridge regression, boosted trees)?
34. Do you think 50 small decision trees are better than a large one? Why?
35. Is actuarial science not a branch of statistics (survival analysis)? If
not, how so?
36. Give examples of data that does not have a Gaussian distribution, nor
log-normal. Give examples of data that has a very chaotic distribution?
37. Why is mean square error a bad measure of model performance? What would
you suggest instead?
38. How can you prove that one improvement you've brought to an algorithm
is really an improvement over not doing anything? Are you familiar with
A/B testing?
39. What is sensitivity analysis? Is it better to have low sensitivity (that
is, great robustness) and low predictive power, or the other way around?
How to perform good cross-validation? What do you think about the idea of
injecting noise in your data set to test the sensitivity of your models?
40. Compare logistic regression w. decision trees, neural networks. How have
these technologies been vastly improved over the last 15 years?
41. Do you know / used data reduction techniques other than PCA? What do you
think of step-wise regression? What kind of step-wise techniques are you
familiar with? When is full data better than reduced data or sample?
42. How would you build non parametric confidence intervals, e.g. for scores?
(see the AnalyticBridge theorem)
43. Are you familiar either with extreme value theory, monte carlo
simulations or mathematical statistics (or anything else) to correctly
estimate the chance of a very rare event?
44. What is root cause analysis? How to identify a cause vs. a correlation?
Give examples.
45. How would you define and measure the predictive power of a metric?
46. How to detect the best rule set for a fraud detection scoring technology?
How do you deal with rule redundancy, rule discovery, and the combinatorial
nature of the problem (for finding optimum rule set - the one with best
predictive power)? Can an approximate solution to the rule set problem be
OK? How would you find an OK approximate solution? How would you decide
it is good enough and stop looking for a better one?
47. How to create a keyword taxonomy?
48. What is a Botnet? How can it be detected?
49. Any experience with using API's? Programming API's? Google or Amazon
API's? AaaS (Analytics as a service)?
50. When is it better to write your own code than using a data science software
package?
51. Which tools do you use for visualization? What do you think of Tableau?
R? SAS? (for graphs). How to efficiently represent 5 dimension in a chart
(or in a video)?
52. What is POC (proof of concept)?
53. What types of clients have you been working with: internal, external,
sales / finance / marketing / IT people? Consulting experience? Dealing
with vendors, including vendor selection and testing?
54. Are you familiar with software life cycle? With IT project life cycle
- from gathering requests to maintenance?
55. What is a cron job?
56. Are you a lone coder? A production guy (developer)? Or a designer
(architect)?
57. Is it better to have too many false positives, or too many false negatives?
58. Are you familiar with pricing optimization, price elasticity, inventory
management, competitive intelligence? Give examples.
59. How does Zillow's algorithm work? (to estimate the value of any home in
US)
60. How to detect bogus reviews, or bogus Facebook accounts used for bad
purposes?
61. How would you create a new anonymous digital currency?
62. Have you ever thought about creating a startup? Around which idea /
concept?
63. Do you think that typed login / password will disappear? How could they
be replaced?
64. Have you used time series models? Cross-correlations with time lags?
Correlograms? Spectral analysis? Signal processing and filtering
techniques? In which context?
65. Which data scientists do you admire most? which startups?
66. How did you become interested in data science?
67. What is an efficiency curve? What are its drawbacks, and how can they
be overcome?
68. What is a recommendation engine? How does it work?
69. What is an exact test? How and when can simulations help us when we do
not use an exact test?
70. What do you think makes a good data scientist?
71. Do you think data science is an art or a science?
72. What is the computational complexity of a good, fast clustering algorithm?
What is a good clustering algorithm? How do you determine the number of
clusters? How would you perform clustering on one million unique keywords,
assuming you have 10 million data points - each one consisting of two
keywords, and a metric measuring how similar these two keywords are? How
would you create this 10 million data points table in the first place?
73. Give a few examples of "best practices" in data science.
74. What could make a chart misleading, difficult to read or interpret? What
features should a useful chart have?
75. Do you know a few "rules of thumb" used in statistical or computer science?
Or in business analytics?
76. What are your top 5 predictions for the next 20 years?
77. How do you immediately know when statistics published in an article (e.g.
newspaper) are either wrong or presented to support the author's point of
view, rather than correct, comprehensive factual information on a specific
subject? For instance, what do you think about the official monthly
unemployment statistics regularly discussed in the press? What could make
them more accurate?
78. Testing your analytic intuition: look at these three charts. Two of them
exhibit patterns. Which ones? Do you know that these charts are called
scatter-plots? Are there other ways to visually represent this type of data?
79. You design a robust non-parametric statistic (metric) to replace
correlation or R square, that (1) is independent of sample size, (2) always
between -1 and +1, and (3) based on rank statistics. How do you normalize
for sample size? Write an algorithm that computes all permutations of n
elements. How do you sample permutations (that is, generate tons
of random permutations) when n is large, to estimate the asymptotic
distribution for your newly created metric? You may use this asymptotic
distribution for normalizing your metric. Do you think that an exact
theoretical distribution might exist, and therefore, we should find it,
and use it rather than wasting our time trying to estimate the asymptotic
distribution using simulations?
80. More difficult, technical question related to previous one. There is an
obvious one-to-one correspondence between permutations of n elements and
integers between 1 and n! Design an algorithm that encodes an integer less
than n! as a permutation of n elements. What would be the reverse algorithm,
used to decode a permutation and transform it back into a number? Hint:
An intermediate step is to use the factorial number
system representation of an integer. Feel free to check this reference
online to answer the question. Even better, feel free to browse the web
to find the full answer to the question (this will test the candidate's
ability to quickly search online and find a solution to a problem without
spending hours reinventing the wheel).
81. How many "useful" votes will a Yelp review receive? My
answer: Eliminate bogus accounts (read this article), or competitor
reviews (how to detect them: use taxonomy to classify users, and location
- two Italian restaurants in same Zip code could badmouth each other and
write great comments for themselves). Detect fake likes: some companies
(e.g. FanMeNow.com) will charge you to produce fake accounts and fake
likes. Eliminate prolific users who like everything, those who hate
everything. Have a blacklist of keywords to filter fake reviews. See if
IP address or IP block of reviewer is in a blacklist such as "Stop Forum
Spam". Create honeypot to catch fraudsters. Also watch out for
disgruntled employees badmouthing their former employer. Watch out for 2
or 3 similar comments posted the same day by 3 users regarding a company
that receives very few reviews. Is it a brand new company? Add more weight
to trusted users (create a category of trusted users). Flag all reviews
that are identical (or nearly identical) and come from same IP address or
same user. Create a metric to measure distance between two pieces of text
(reviews). Create a review or reviewer taxonomy. Use hidden decision
trees to rate or score review and reviewers.
82. What did you do today? Or what did you do this week / last week?
83. What/when is the latest data mining book / article you read? What/when
is the latest data mining conference / webinar / class / workshop / training
you attended? What/when is the most recent programming skill that you
acquired?
84. What are your favorite data science websites? Who do you admire most in
the data science community, and why? Which company do you admire most?
85. What/when/where is the last data science blog post you wrote?
86. In your opinion, what is data science? Machine learning? Data mining?