(Failure of OOD detection under invariant classifier) Consider an out-of-distribution input which contains the environmental feature: ? out ( x ) = M inv z out + M e z e , where z out ? ? inv . Given the invariant classifier (cf. Lemma 2), the posterior probability for the OOD input is p ( y = 1 ? ? out ) = ? ( 2 p ? z e ? + log ? / ( 1 ? ? ) ) , where ? is the logistic function. Thus for arbitrary confidence 0 < c : = P ( y = 1 ? ? out ) < 1 , there exists ? out ( x ) with z e such that p ? z e = 1 2 ? log c ( 1 ? ? ) ? ( 1 ? c ) .
Facts. Envision an out-of-delivery input x out having Yards inv = [ We s ? s 0 1 ? s ] , and you will Yards e = [ 0 s ? e p ? ] , then your ability icon was ? age ( x ) = [ z away p ? z e ] , where p is the product-standard vector outlined when you look at the Lemma dos .
Then we have P ( y = 1 ? ? out ) = P ( y = 1 ? z out , p ? z e ) = ? ( 2 p ? z e ? + log ? / ( 1 ? ? ) ) , where ? is the logistic function. Thus for arbitrary confidence 0 < c : = P ( y = 1 ? ? out ) < 1 , there exists ? out ( x ) with z e such that p ? z e = 1 2 ? log c ( 1 ? ? ) ? ( 1 ? c ) . ?
Remark: During the a far more general circumstances, z out might be modeled because an arbitrary vector that is independent of the from inside the-delivery names y = step one and y = ? step one and ecological has: z aside ? ? y and you may z aside ? ? z e . Hence inside Eq. 5 i have P ( z aside ? y = step 1 ) = P ( z aside ? y = ? step one ) = P ( z out ) . Up coming P ( y = step one ? ? aside ) = ? ( dos p ? z age ? + diary ? / ( step one ? ? ) ) , same as from inside the Eq. 7 . Hence all of our fundamental theorem still keeps less than alot more general instance.
Appendix B Extension: Color Spurious Correlation
To further confirm all of our conclusions beyond background and you will gender spurious (environmental) has, we provide even more fresh efficiency into ColorMNIST dataset, since the found inside the Shape 5 .
Evaluation Activity step 3: ColorMNIST.
[ lecun1998gradient ] , which composes colored backgrounds on digit images. In this dataset, E = < red>denotes the background color and we use Y = < 0>as in-distribution classes. The correlation between the background color e and the digit y is explicitly controlled, with r ? < 0.25>. That is, r denotes the probability of P ( e = red ? y = 0 ) = P ( e = purple ? y = 0 ) = P ( e = green ? y = 1 ) = P ( e = pink ? y = 1 ) , while 0.5 ? r = P ( e = green ? y = 0 ) = P ( e = pink ? y = 0 ) = P ( e = red ? y = 1 ) = P ( e = purple ? y = 1 ) . Note that the maximum correlation r (reported in Table 4 ) is 0.45 . As ColorMNIST is relatively simpler compared to Waterbirds and CelebA, further increasing the correlation results flirt in less interesting environments where the learner can easily pick up the contextual information. For spurious OOD, we use digits < 5>with background color red and green , which contain overlapping environmental features as the training data. For non-spurious OOD, following common practice [ MSP ] , we use the Textures [ cimpoi2014describing ] , LSUN [ lsun ] and iSUN [ xu2015turkergaze ] datasets. We train on ResNet-18 [ he2016deep ] , which achieves 99.9 % accuracy on the in-distribution test set. The OOD detection performance is shown in Table 4 .