For which of the following problems would anomaly detection be a suitable algorithm?
From a large set of primary care patient records, identify individuals who might have unusual health conditions.
Since you are just looking for unusual conditions instead of a particular disease, this is a good application of anomaly detection.
Given data from credit card transactions, classify each transaction according to type of purchase (for example: food, transportation, clothing).
Given an image of a face, determine whether or not it is the face of a particular famous individual.
Given a dataset of credit card transactions, identify unusual transactions to flag them as possibly fraudulent.
By modeling “normal” credit card transactions, you can then use anomaly detection to flag the unusuals ones which might be fraudulent.
In a computer chip fabrication plant, identify microchips that might be defective.
The defective chips are the anomalies you are looking for by modeling the properties of non-defective chips.
From a large set of hospital patient records, predict which patients have a particular disease (say, the flu).
Suppose you have trained an anomaly detection system for fraud detection, and your system that flags anomalies when p(x) is less than ε, and you find on the cross-validation set that it is missing many fradulent transactions (i.e., failing to flag them as anomalies). What should you do?
Increase ε
By increasing ε, you will flag more anomalies, as desired.
Decrease ε
Suppose you have trained an anomaly detection system for fraud detection, and your system that flags anomalies when p(x) is less than ε, and you find on the cross-validation set that it is mis-flagging far too many good transactions as fradulent. What should you do?
Increase ε
Decrease ε
By decreasing ε, you will flag fewer anamolies, as desired.
Suppose you are developing an anomaly detection system to catch manufacturing defects in airplane engines. You model uses You have two features = vibration intensity, and = heat generated. Both and take on values between 0 and 1 (and are strictly greater than 0), and for most “normal” engines you expect that . One of the suspected anomalies is that a flawed engine may vibrate very intensely even without generating much heat (large , small ), even though the particular values of and may not fall outside their typical ranges of values. What additional feature should you create to capture these types of anomalies:
This is correct, as it will take on large values for anomalous examples and smaller values for normal examples.
Which of the following are true? Check all that apply.
If you do not have any labeled data (or if all your data has label y = 0), then is is still possible to learn p(x), but it may be harder to evaluate the system or choose a good value of ϵ.
Only negative examples are used in training, but it is good to have some labeled data of both types for cross-validation.
If you are developing an anomaly detection system, there is no way to make use of labeled data to improve your system.
When choosing features for an anomaly detection system, it is a good idea to look for features that take on unusually large or small values for (mainly the) anomalous examples.
These are good features, as they will lie outside the learned model, so you will have small values for p(x) with these examples.
If you have a large labeled training set with many positive examples and many negative examples, the anomaly detection algorithm will likely perform just as well as a supervised learning algorithm such as an SVM.
In a typical anomaly detection setting, we have a large number of anomalous examples, and a relatively small number of normal/non-anomalous examples.
When developing an anomaly detection system, it is often useful to select an appropriate numerical performance metric to evaluate the effectiveness of the learning algorithm.
You should have a good evaluation metric, so you can evaluate changes to the model such as new features.
In anomaly detection, we fit a model p(x) to a set of negative ( y=0) examples, without using any positive examples we may have collected of previously observed anomalies.
We want to model “normal” examples, so we only use negative examples in training.
When evaluating an anomaly detection algorithm on the cross validation set (containing some positive and some negative examples), classification accuracy is usually a good evaluation metric to use.
You have a 1-D dataset and you want to detect outliers in the dataset. You first plot the dataset and it looks like this: Suppose you fit the gaussian distribution parameters and to this dataset. Which of the following values for and might you get?
-3, 4
This is correct, as the data are centered around -3 and tail most of the points lie in [-5, -1].
= vibration intensity, and
= heat generated. Both
. One of the suspected anomalies is that a flawed engine may vibrate very intensely even without generating much heat (large
should you create to capture these types of anomalies:
and you want to detect outliers in the dataset. You first plot the dataset and it looks like this:
and
to this dataset.
-3,
4
-6,
4
No comments