BSI 24/30456154 DC:2024 Edition
$13.70
BS EN IEC 63450 Testing of Artificial Intelligence / Machine Learning-enabled Medical Devices
| Published By | Publication Date | Number of Pages |
| BSI | 2024 | 39 |
PDF Catalog
| PDF Pages | PDF Title |
|---|---|
| 6 | FOREWORD |
| 8 | INTRODUCTION |
| 9 | 1 Scope 2 Normative references 3 Terms and definitions |
| 12 | 3.22 Terms specific to Annex A 4 Test planning 4.1 Test plan |
| 13 | 4.2 Test objectives 4.3 Test scope 4.4 Responsibilities and roles |
| 14 | 4.5 Test environment 4.5.1 General 4.5.2 Test tools 4.5.3 Test data 4.6 Deliverables 4.7 Re-testing after changes |
| 15 | 4.8 Test methods 4.8.1 General 4.8.2 Test oracle 4.8.3 Cross-Validation testing 4.8.3.1 Test method description 4.8.3.2 General 4.8.3.3 Holdout Cross-Validation |
| 16 | 4.8.3.4 k-Fold Cross-Validation |
| 17 | 4.8.3.5 Repeated k-folds Cross-Validation |
| 18 | 4.8.3.6 Stratified k-fold Cross-Validation |
| 19 | 4.8.3.7 Time Series Cross-Validation |
| 20 | 4.8.3.8 Test method use cases 4.8.3.9 Test method specific requirements |
| 21 | 4.8.4 Black-box test methods 4.8.4.1 Combinatorial testing 4.8.4.2 Test method description |
| 22 | 4.8.4.3 Test method use cases 4.8.4.4 Test method specific requirements 4.8.4.5 Back-to-back testing 4.8.4.6 Test method description 4.8.4.7 Test method use cases |
| 23 | 4.8.4.8 Test method specific requirements 4.8.4.9 Extended back-to-back testing 4.8.4.10 Test method description 4.8.4.11 Test method use cases 4.8.4.12 Test method specific requirements 4.8.4.13 A/B testing 4.8.4.14 Test method description |
| 24 | 4.8.4.15 Test method use cases 4.8.4.16 Test method specific requirements 4.8.4.17 Metamorphic testing 4.8.4.18 Test method description 4.8.4.19 Test method use cases 4.8.4.20 Test method specific requirements |
| 25 | 4.8.4.21 Exploratory testing 4.8.4.22 Test method description 4.8.4.23 Test method use cases 4.8.4.24 Test method specific requirements 4.8.5 White-box test methods |
| 26 | 4.8.6 AI explainability/transparency 4.8.6.1 Explainability 4.8.6.2 LIME |
| 27 | 4.8.6.3 SHAP 4.8.6.4 Transparency |
| 28 | 4.9 Statistics 4.9.1 General 4.9.2 Classification metrics 4.9.3 Regression metrics |
| 29 | 4.10 Test phases / stages 5 Test specification 5.1 Test identification 5.2 Test conditions |
| 30 | 5.3 Acceptance criteria 5.4 Test data 5.4.1 Test data selection 5.4.2 Data relevance 5.4.3 Data representativeness 5.4.4 Test data quantification 5.4.5 Test data preparation |
| 31 | 5.4.6 DOUP (Data Of Unknown Provenance) 5.4.7 Artificial data 5.4.8 Test data labelling |
| 32 | 5.4.9 Use of unlabelled data 5.4.10 Traceability 5.4.10.1 Test data identification 5.4.10.2 Test data usage logging 6 Test results 6.1 Test results documentation 6.2 Test deviations |
| 33 | Annex A (normative) Continuous learning A.1 Scope A.2 General concepts and requirements |
| 34 | A.3 Control mechanisms A.3.1 Pre-testing by the manufacturer A.3.2 Evaluation of training algorithm responsiveness |
| 35 | A.3.3 Limited learning A.3.4 Evaluation of change A.3.5 Utilization of back-to-back testing A.3.6 Released model as a test oracle A.4 New data samples A.4.1 Training data A.4.2 Test data |
| 37 | Annex B (informative) Data/information leakage B.1 Risks of data/information leakage and impact on resulting AI systems B.2 Sources of data/information leakage B.3 The ideal test set B.4 How to avoid data/information leakage |
| 39 | Bibliography |
