Evaluation Results - Version 1

We compute quality measures with respect to the four ground truth seismic cubes depending on their discrete or continuous amplitude values. The metrics are summarized below, for more precise mathematical details, please refer to our paper [1].

Discrete group:

  • precision: Percentage of true positives of extracted edges/faults in attributes to all extracted edges/faults.
  • recall: Percentage of true positives of extracted edges/faults in attributes to all ground truth edges/faults.
  • rmseDistance: Root mean square error weighted by the nonlinear distance (sigmf([1:1:30],[0.5 10])) between edges/faults in attributes and their closest edges/faults in ground truth. Extracted edges/faults using the seismic attributes are treated differently depending on their distance to the closest object in the ground truth.

Continuous group:

  • recall: Percentage of true positives of extracted attributes to ground truth.
  • rmse: Root mean square error between attributes and ground truth.
  • rmseDiscontinuity: Root mean square error between attributes and ground truth without including the edges/faults area. This originates from the fact that most structural attributes (except edge attributes) are undefined at (or very near) edges or discontinuities. This is because derivatives are undefined at discontinuities.

The threshold for a true positive is different for each ground truth category. For each continuous group, the threshold D is 20% of the difference between the maximum and minimum of its ground truth. If the submitted seismic attribute for edge/fault is continuous, not discrete, they will be discretized according to the top 20% rule.

We have computed their varieties, both in 3D and 2D, as well as in different dip regions [0°, 90°] and [0°, 45°]. In the 3D version, the quality measures are computed over the entire seismic cube, whereas in 2D they are first computed on every inline, crossline, and time section of the cube, followed by a mean operation. A caveat of the above metrics is that they do not discriminate between attributes at different dip angle ranges. However, in practice, it is very hard to image structural dips beyond 45 degrees for most types of seismic data. Therefore, more quality metrics (named *_LowDip), which account only for the dip in the 0-45 degrees range, are developed.

Submit and evaluate your own attributes.

Edge/Fault
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.0311 0.0254 0.0299 0.0259 0.0296 0.0313 0.0301 0.0352 0.0372 0.0356
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.0369 0.0329 0.0268 0.0258 0.0411 0.0328 0.0314 0.0579 0.0428 0.0408
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.0085 0.0046 0.0040 0.0045 0.0077 0.0070 0.0077 0.0142 0.0131 0.0140
SLB_Petrel_Chaos_Xing 2016/12/11 0.0870 0.0717 0.0676 0.0577 0.0897 0.0835 0.0720 0.1249 0.1164 0.0997
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.0313 0.0372 0.0157 0.0162 0.0470 0.0205 0.0213 0.0632 0.0301 0.0308
SLB_Petrel_Var_Xing 2016/12/11 0.0554 0.0423 0.0376 0.0399 0.0545 0.0489 0.0513 0.0784 0.0717 0.0742
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.3225 0.3490 0.4114 0.3564 0.3190 0.3370 0.3241 0.2629 0.2772 0.2654
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.3739 0.4523 0.3683 0.3550 0.4430 0.3528 0.3386 0.4323 0.3190 0.3042
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.0809 0.0632 0.0551 0.0623 0.0824 0.0751 0.0826 0.1057 0.0977 0.1043
SLB_Petrel_Chaos_Xing 2016/12/11 0.8771 0.9862 0.9291 0.7934 0.9663 0.8995 0.7753 0.9316 0.8686 0.7440
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.6524 0.6576 0.5879 0.6705 0.6729 0.6032 0.6950 0.6612 0.6206 0.7026
SLB_Petrel_Var_Xing 2016/12/11 0.5541 0.5818 0.5173 0.5485 0.5870 0.5263 0.5524 0.5848 0.5349 0.5537
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.1817 0.2357 0.2266 0.2329 0.1839 0.1835 0.1821 0.1295 0.1294 0.1316
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.2132 0.2480 0.2532 0.2510 0.2095 0.2154 0.2134 0.1730 0.1775 0.1778
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.2841 0.3374 0.3410 0.3382 0.2822 0.2857 0.2837 0.2288 0.2302 0.2298
SLB_Petrel_Chaos_Xing 2016/12/11 0.1880 0.2196 0.2151 0.2627 0.1793 0.1733 0.2119 0.1441 0.1385 0.1477
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.3976 0.3506 0.5334 0.5465 0.2821 0.4485 0.4563 0.2249 0.3638 0.3722
SLB_Petrel_Var_Xing 2016/12/11 0.2494 0.2942 0.2991 0.3008 0.2393 0.2450 0.2468 0.2056 0.2070 0.2072
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.0279 0.0219 0.0285 0.0225 0.0261 0.0286 0.0264 0.0314 0.0342 0.0316
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.0397 0.0370 0.0244 0.0246 0.0503 0.0320 0.0329 0.0702 0.0425 0.0433
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.0196 0.0159 0.0122 0.0107 0.0186 0.0153 0.0141 0.0349 0.0293 0.0258
SLB_Petrel_Chaos_Xing 2016/12/11 0.0870 0.0695 0.0651 0.0575 0.0897 0.0824 0.0722 0.1272 0.1182 0.1012
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.0487 0.0807 0.0224 0.0188 0.0879 0.0278 0.0237 0.1058 0.0382 0.0334
SLB_Petrel_Var_Xing 2016/12/11 0.0999 0.0891 0.0787 0.0762 0.1010 0.0917 0.0887 0.1322 0.1225 0.1186
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.2761 0.2959 0.3489 0.3009 0.2733 0.2875 0.2759 0.2296 0.2419 0.2311
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.3435 0.4128 0.3305 0.3166 0.4120 0.3224 0.3073 0.4102 0.2978 0.2823
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.1020 0.0889 0.0774 0.0865 0.1035 0.0944 0.1036 0.1249 0.1155 0.1230
SLB_Petrel_Chaos_Xing 2016/12/11 0.8735 0.9769 0.9270 0.7975 0.9577 0.8922 0.7756 0.9271 0.8644 0.7434
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.6886 0.7029 0.6310 0.7149 0.7114 0.6382 0.7362 0.6867 0.6466 0.7294
SLB_Petrel_Var_Xing 2016/12/11 0.5758 0.6117 0.5527 0.5795 0.6006 0.5446 0.5687 0.6002 0.5540 0.5707
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.3036 0.3262 0.3119 0.3253 0.3090 0.3052 0.3085 0.2832 0.2792 0.2839
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.3403 0.3474 0.3661 0.3624 0.3306 0.3508 0.3476 0.3040 0.3283 0.3257
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.3703 0.3853 0.3878 0.3885 0.3700 0.3727 0.3733 0.3499 0.3521 0.3528
SLB_Petrel_Chaos_Xing 2016/12/11 0.2815 0.2855 0.3107 0.3450 0.2562 0.2768 0.3168 0.2205 0.2407 0.2813
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.5500 0.3713 0.6593 0.6998 0.3511 0.6326 0.6729 0.3322 0.5967 0.6343
SLB_Petrel_Var_Xing 2016/12/11 0.3443 0.3621 0.3668 0.3691 0.3396 0.3453 0.3484 0.3178 0.3234 0.3260
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.0470 0.0401 0.0476 0.0404 0.0455 0.0488 0.0456 0.0504 0.0541 0.0505
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.0796 0.0920 0.0613 0.0572 0.0986 0.0675 0.0629 0.1212 0.0804 0.0750
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.2839 0.2754 0.2432 0.2613 0.2833 0.2503 0.2736 0.3441 0.2929 0.3307
SLB_Petrel_Chaos_Xing 2016/12/11 0.1339 0.1175 0.1143 0.1121 0.1350 0.1274 0.1231 0.1726 0.1564 0.1463
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.1408 0.1883 0.1311 0.0714 0.1922 0.1395 0.0781 0.2103 0.1617 0.0943
SLB_Petrel_Var_Xing 2016/12/11 0.4523 0.4657 0.4528 0.4186 0.4493 0.4431 0.4140 0.4859 0.4839 0.4577
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.4934 0.4900 0.5821 0.5025 0.4930 0.5227 0.5030 0.4378 0.4642 0.4457
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.4746 0.5430 0.4580 0.4450 0.5388 0.4519 0.4392 0.5430 0.4317 0.4203
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.0082 0.0074 0.0069 0.0093 0.0075 0.0069 0.0095 0.0082 0.0072 0.0107
SLB_Petrel_Chaos_Xing 2016/12/11 0.8862 1.0000 0.9254 0.7627 0.9968 0.9201 0.7694 0.9647 0.8940 0.7424
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.5367 0.5537 0.4762 0.5678 0.5639 0.4888 0.5767 0.5446 0.4839 0.5745
SLB_Petrel_Var_Xing 2016/12/11 0.4621 0.4752 0.3951 0.4390 0.5164 0.4398 0.4783 0.5086 0.4373 0.4696
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.1128 0.1384 0.1424 0.1402 0.1073 0.1032 0.1099 0.0922 0.0868 0.0948
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.0943 0.0837 0.1118 0.1177 0.0748 0.0998 0.1064 0.0671 0.0909 0.0970
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.0111 0.0105 0.0112 0.0116 0.0107 0.0113 0.0117 0.0103 0.0112 0.0114
SLB_Petrel_Chaos_Xing 2016/12/11 0.0638 0.0553 0.0948 0.1053 0.0432 0.0598 0.0817 0.0308 0.0404 0.0626
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.0995 0.0740 0.0961 0.1586 0.0659 0.0852 0.1470 0.0590 0.0763 0.1334
SLB_Petrel_Var_Xing 2016/12/11 0.0077 0.0025 0.0058 0.0169 0.0026 0.0055 0.0158 0.0025 0.0051 0.0129
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.0763 0.0679 0.0827 0.0682 0.0714 0.0787 0.0759 0.0791 0.0844 0.0786
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.1207 0.1370 0.1034 0.1064 0.1414 0.1078 0.1091 0.1572 0.1122 0.1119
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.3929 0.3625 0.3552 0.3739 0.3618 0.3571 0.3739 0.4746 0.4332 0.4441
SLB_Petrel_Chaos_Xing 2016/12/11 0.1544 0.1358 0.1412 0.1448 0.1533 0.1512 0.1536 0.1768 0.1670 0.1658
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.1808 0.2451 0.1770 0.1049 0.2455 0.1828 0.1110 0.2494 0.1951 0.1165
SLB_Petrel_Var_Xing 2016/12/11 0.4840 0.5091 0.4807 0.4352 0.4933 0.4868 0.4572 0.5068 0.5063 0.4805
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.4649 0.4526 0.5390 0.4610 0.4618 0.4881 0.4672 0.4295 0.4529 0.4315
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.4504 0.5338 0.4420 0.4266 0.5126 0.4213 0.4047 0.5202 0.4036 0.3888
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.0113 0.0107 0.0098 0.0127 0.0104 0.0096 0.0125 0.0115 0.0106 0.0134
SLB_Petrel_Chaos_Xing 2016/12/11 0.9056 1.0000 0.9433 0.8136 0.9971 0.9405 0.8251 0.9445 0.9008 0.7853
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.5734 0.5874 0.5086 0.6028 0.6118 0.5330 0.6255 0.5698 0.5107 0.6107
SLB_Petrel_Var_Xing 2016/12/11 0.4987 0.5104 0.4308 0.4769 0.5628 0.4902 0.5294 0.5318 0.4624 0.4940
Attribute Name Updated Average Train
N/F
Train
N/R
Train
N/C
Test 1
N/F
Test 1
N/R
Test 1
N/C
Test 2
N/F
Test 2
N/R
Test 2
N/C
DGB_OD_FaultSim_Harishidayat 2016/11/17 0.3226 0.3502 0.3449 0.3561 0.3226 0.3141 0.3285 0.2971 0.2870 0.3024
DGB_OD_InstaAmp_Harishidayat 2016/11/17 0.2380 0.1752 0.2905 0.2973 0.1621 0.2734 0.2805 0.1495 0.2517 0.2617
SLB_Petrel_Amplitude contrast_Xing 2016/12/11 0.0101 0.0091 0.0106 0.0110 0.0092 0.0106 0.0110 0.0086 0.0103 0.0105
SLB_Petrel_Chaos_Xing 2016/12/11 0.2073 0.2106 0.2561 0.2619 0.1703 0.2160 0.2393 0.1201 0.1768 0.2145
SLB_Petrel_ResidualConsistentDip_Xing 2016/12/11 0.2251 0.1134 0.2172 0.3743 0.1088 0.2070 0.3621 0.1071 0.1955 0.3404
SLB_Petrel_Var_Xing 2016/12/11 0.0173 0.0067 0.0104 0.0336 0.0070 0.0104 0.0326 0.0106 0.0132 0.0315
Ground Truth vs Attribute Result (Snapshots from different view angles. Please single click the table cell above.)