{"id":474,"date":"2021-07-12T09:18:10","date_gmt":"2021-07-12T09:18:10","guid":{"rendered":"https:\/\/infosecml.com\/?p=474"},"modified":"2021-11-01T07:34:55","modified_gmt":"2021-11-01T07:34:55","slug":"bayes-theorem-getting-a-better-understanding-of-probability","status":"publish","type":"post","link":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/","title":{"rendered":"Bayes Theorem – Getting a better understanding of Probability"},"content":{"rendered":"\n
All Probabilities Are Conditional<\/h3>\n\n\n\n
Even when statisticians tell you that the probability of some event is A what they are actually saying is that given all of the background the probability is A. Something as simple as rolling a six sided die gives you a 1\/6th probability of any individual number, that is conditional on the regularity of the die shape, how it is weighted, how it is rolled etc. We just tend to take this background context for granted and forget that it is there. This is normally fine for the rolls of dice, but it can be problematic when we are dealing with probabilities where the context is less easily ignored. Thomas Bayes was an 18th century minister and statistician who developed a way of taking the known context of a probability into account when calculating the probability itself.<\/p>\n\n\n\n
To understand Bayes, we need to define a few simple terms.<\/p>\n\n\n\n
The probability of A is written as P(A)<\/strong> which we call the marginal probability or the prior probability in Bayes theorem. That is, the probability of an event irrespective of any external context.<\/p>\n\n\n\n
In addition, the joint probability, the probability of two simultaneous events, such as throwing two sixes is written P(A, B). We say that is is the probability of A and B.<\/p>\n\n\n\n
The probability of an event given some context is written P(A|B)<\/strong> \u2013 The probability of A given B. We call this the conditional probability. One very important thing to note is that P(A|B)<\/strong> is not the same as P(B|A)<\/strong>. The probability that your house is on fire given that you see smoke is not the same as the probability you see smoke given that your house is on fire. This is one of the most common mistakes made in probability, even it would seem, by trained statisticians.<\/p>\n\n\n\n
Th conditional probability is given by:<\/p>\n\n\n\n
P(A|B) = P(B|A) * P(A) \/ P(B)<\/strong> \u2013 This is Bayes theorem<\/p>\n\n\n\n
The Probability of (A given B) is the probability of (B given A) times the probability of A divided by the probability of B.<\/p>\n\n\n\n
By now you are probably wondering how this is useful and relevant to information security so let\u2019s take a look at a simple though probably somewhat contrived example.<\/p>\n\n\n\n
Let us suppose you are trying to improve detection of bad behaviour of accounts in your environment, either because they have been compromised or because a user is a threat. You decide to look at failed remote logins as an indicator and looking back through years of logs you discover that multiple failed remote logins on linux systems might be an indicator of a compromised account.<\/p>\n\n\n\n
Looking further you notice:<\/p>\n\n\n\n
0.5% of accounts have some sort of compromise or are being misused by employees, which means that 99.5% are not compromised.<\/p>\n\n\n\n
Where an account is compromised, 80% of the time that is detectable through multiple failed remote logins, and therefore 20% of those compromised accounts can not be detected in that way.<\/p>\n\n\n\n
11% of accounts which are not compromised also generate these multiple failed logins, Which means that failed logins is a good indicator of a compromised account 89% of the time.<\/p>\n\n\n\n
So if we see multiple failed login what is the probability that the account if compromised? We can use Bayes to get a more accurate picture of this:<\/p>\n\n\n\n
If we see multiple failed logins for an account it could be a real result or it could be a false positive. The chance that it is a true positive result is given by 0.5% * 80% = 0.004.<\/p>\n\n\n\n
The chance of a false positive is 99.5% * 11% = 0.10945.<\/p>\n\n\n\n
Using Bayes theorem<\/h3>\n\n\n\n
Bayes tells us that the probability of a hypothesis being correct given the evidence P(H|E)<\/strong>, is the probability of seeing the evidence given the hypothesis P(E|H)<\/strong>, multiplied by your initial probability of the hypothesis being correct, the prior probability, P(H)<\/strong>, divided by the probability of seeing the evidence P(E)<\/strong>.<\/p>\n\n\n\n
The chance this is a compromised account given the multiple failed logins is give by:<\/p>\n\n\n\n
P(H|E) <\/strong>= Chance of it being a compromised account given detections of multiple failed logins<\/p>\n\n\n\n
P(E|H)<\/strong> = Chance of failed logins given that it is a compromised account = 80%<\/p>\n\n\n\n
P(H)<\/strong> = Chance of it being a compromised account = 0.5%<\/p>\n\n\n\n
P(not H)<\/strong> = Chance of it not being a compromised account = 99.5%<\/p>\n\n\n\n
P(E|not H)<\/strong> = Chance of seeing failed logins from a non-compromised account = 10.945%<\/p>\n\n\n\n
So the probability of seeing the evidence is given by:<\/p>\n\n\n\n
Where E represents evidence and H is the hypothesis.<\/p>\n\n\n\n<\/figure>\n\n\n\n
And plugging in the numbers we get:<\/p>\n\n\n\n<\/figure>\n\n\n\n
Which gives us a chance of 3.54%<\/strong> that when we see multiple failed remote logins that the account attempting them is in some way compromised. Was that the number you were expecting? Of course what you now do with that probability is up to you.<\/p>\n\n\n\n
In machine learning you will probably hear about Naive Bayes, which is a set of supervised learning algorithms which use Bayes theorem with a \u201cnaive\u201d assumption of conditional independence \u2013 That is that conditions are independent and not driven by other elements. You may also encounter Bayesian networks where probabilities are lined together with conditional probabilities flowing back through the network.<\/p>\n\n\n\n
Bayes theorem on its own is a good way to ensure you are thinking properly about probabilities and not ignoring the conditions around the probabilities your are measuring and estimating.<\/p>\n","protected":false},"excerpt":{"rendered":"
All Probabilities Are Conditional Even when statisticians tell you that the probability of some event is A what they are actually saying is that given all of the background the probability is A. Something as simple as rolling a six sided die gives you a 1\/6th probability of any individual number, that is conditional on […]<\/p>\n","protected":false},"author":1,"featured_media":473,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"class_list":["post-474","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning"],"yoast_head":"\n
Bayes Theorem - Getting a better understanding of Probability - InfoSecML<\/title>\n<meta name=\"description\" content=\"Understanding Bayes Theorem is essential when considering probabilities that may be influenced by some existing context\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Bayes Theorem - Getting a better understanding of Probability - InfoSecML\" \/>\n<meta property=\"og:description\" content=\"Understanding Bayes Theorem is essential when considering probabilities that may be influenced by some existing context\" \/>\n<meta property=\"og:url\" content=\"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/\" \/>\n<meta property=\"og:site_name\" content=\"InfoSecML\" \/>\n<meta property=\"article:published_time\" content=\"2021-07-12T09:18:10+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-11-01T07:34:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/infosecml.com\/wp-content\/uploads\/2021\/07\/Thomas_Bayes.gif\" \/>\n\t<meta property=\"og:image:width\" content=\"304\" \/>\n\t<meta property=\"og:image:height\" content=\"326\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/gif\" \/>\n<meta name=\"author\" content=\"Steve Gailey\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Steve Gailey\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/\"},\"author\":{\"name\":\"Steve Gailey\",\"@id\":\"https:\\\/\\\/infosecml.com\\\/#\\\/schema\\\/person\\\/f11f4ea3133147a580202f13b6da27e8\"},\"headline\":\"Bayes Theorem – Getting a better understanding of Probability\",\"datePublished\":\"2021-07-12T09:18:10+00:00\",\"dateModified\":\"2021-11-01T07:34:55+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/\"},\"wordCount\":919,\"publisher\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/#\\\/schema\\\/person\\\/f11f4ea3133147a580202f13b6da27e8\"},\"image\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/infosecml.com\\\/wp-content\\\/uploads\\\/2021\\\/07\\\/Thomas_Bayes.gif\",\"articleSection\":[\"Machine Learning\"],\"inLanguage\":\"en-GB\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/\",\"url\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/\",\"name\":\"Bayes Theorem - Getting a better understanding of Probability - InfoSecML\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/infosecml.com\\\/wp-content\\\/uploads\\\/2021\\\/07\\\/Thomas_Bayes.gif\",\"datePublished\":\"2021-07-12T09:18:10+00:00\",\"dateModified\":\"2021-11-01T07:34:55+00:00\",\"description\":\"Understanding Bayes Theorem is essential when considering probabilities that may be influenced by some existing context\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/#primaryimage\",\"url\":\"https:\\\/\\\/infosecml.com\\\/wp-content\\\/uploads\\\/2021\\\/07\\\/Thomas_Bayes.gif\",\"contentUrl\":\"https:\\\/\\\/infosecml.com\\\/wp-content\\\/uploads\\\/2021\\\/07\\\/Thomas_Bayes.gif\",\"width\":304,\"height\":326,\"caption\":\"Thomas Bayes\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/infosecml.com\\\/index.php\\\/2021\\\/07\\\/12\\\/bayes-theorem-getting-a-better-understanding-of-probability\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/infosecml.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Bayes Theorem – Getting a better understanding of Probability\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/infosecml.com\\\/#website\",\"url\":\"https:\\\/\\\/infosecml.com\\\/\",\"name\":\"InfoSecML\",\"description\":\"The home of Machine Learning and Advanced Analytics for Information Security\",\"publisher\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/#\\\/schema\\\/person\\\/f11f4ea3133147a580202f13b6da27e8\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/infosecml.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/infosecml.com\\\/#\\\/schema\\\/person\\\/f11f4ea3133147a580202f13b6da27e8\",\"name\":\"Steve Gailey\",\"logo\":{\"@id\":\"https:\\\/\\\/infosecml.com\\\/#\\\/schema\\\/person\\\/image\\\/\"},\"sameAs\":[\"http:\\\/\\\/infosecml.com\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Bayes Theorem - Getting a better understanding of Probability - InfoSecML","description":"Understanding Bayes Theorem is essential when considering probabilities that may be influenced by some existing context","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/","og_locale":"en_GB","og_type":"article","og_title":"Bayes Theorem - Getting a better understanding of Probability - InfoSecML","og_description":"Understanding Bayes Theorem is essential when considering probabilities that may be influenced by some existing context","og_url":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/","og_site_name":"InfoSecML","article_published_time":"2021-07-12T09:18:10+00:00","article_modified_time":"2021-11-01T07:34:55+00:00","og_image":[{"width":304,"height":326,"url":"https:\/\/infosecml.com\/wp-content\/uploads\/2021\/07\/Thomas_Bayes.gif","type":"image\/gif"}],"author":"Steve Gailey","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Steve Gailey","Estimated reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/#article","isPartOf":{"@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/"},"author":{"name":"Steve Gailey","@id":"https:\/\/infosecml.com\/#\/schema\/person\/f11f4ea3133147a580202f13b6da27e8"},"headline":"Bayes Theorem – Getting a better understanding of Probability","datePublished":"2021-07-12T09:18:10+00:00","dateModified":"2021-11-01T07:34:55+00:00","mainEntityOfPage":{"@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/"},"wordCount":919,"publisher":{"@id":"https:\/\/infosecml.com\/#\/schema\/person\/f11f4ea3133147a580202f13b6da27e8"},"image":{"@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/#primaryimage"},"thumbnailUrl":"https:\/\/infosecml.com\/wp-content\/uploads\/2021\/07\/Thomas_Bayes.gif","articleSection":["Machine Learning"],"inLanguage":"en-GB"},{"@type":"WebPage","@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/","url":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/","name":"Bayes Theorem - Getting a better understanding of Probability - InfoSecML","isPartOf":{"@id":"https:\/\/infosecml.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/#primaryimage"},"image":{"@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/#primaryimage"},"thumbnailUrl":"https:\/\/infosecml.com\/wp-content\/uploads\/2021\/07\/Thomas_Bayes.gif","datePublished":"2021-07-12T09:18:10+00:00","dateModified":"2021-11-01T07:34:55+00:00","description":"Understanding Bayes Theorem is essential when considering probabilities that may be influenced by some existing context","breadcrumb":{"@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/#primaryimage","url":"https:\/\/infosecml.com\/wp-content\/uploads\/2021\/07\/Thomas_Bayes.gif","contentUrl":"https:\/\/infosecml.com\/wp-content\/uploads\/2021\/07\/Thomas_Bayes.gif","width":304,"height":326,"caption":"Thomas Bayes"},{"@type":"BreadcrumbList","@id":"https:\/\/infosecml.com\/index.php\/2021\/07\/12\/bayes-theorem-getting-a-better-understanding-of-probability\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/infosecml.com\/"},{"@type":"ListItem","position":2,"name":"Bayes Theorem – Getting a better understanding of Probability"}]},{"@type":"WebSite","@id":"https:\/\/infosecml.com\/#website","url":"https:\/\/infosecml.com\/","name":"InfoSecML","description":"The home of Machine Learning and Advanced Analytics for Information Security","publisher":{"@id":"https:\/\/infosecml.com\/#\/schema\/person\/f11f4ea3133147a580202f13b6da27e8"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/infosecml.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":["Person","Organization"],"@id":"https:\/\/infosecml.com\/#\/schema\/person\/f11f4ea3133147a580202f13b6da27e8","name":"Steve Gailey","logo":{"@id":"https:\/\/infosecml.com\/#\/schema\/person\/image\/"},"sameAs":["http:\/\/infosecml.com"]}]}},"_links":{"self":[{"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/posts\/474","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/comments?post=474"}],"version-history":[{"count":6,"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/posts\/474\/revisions"}],"predecessor-version":[{"id":499,"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/posts\/474\/revisions\/499"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/media\/473"}],"wp:attachment":[{"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/media?parent=474"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/categories?post=474"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infosecml.com\/index.php\/wp-json\/wp\/v2\/tags?post=474"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"\"](\"https:\/\/infosecml.com\/wp-content\/uploads\/2021\/07\/Bayes-1.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/infosecml.com\/wp-content\/uploads\/2021\/07\/Bayes_worked.png\")
<\/figure>\n\n\n\n