Explanatory Machine Learning

Description
Research projects
Related Publications
Invited lectures&talks
Media
Student projects supervised

 

Description

The search for explanations in the field of machine learning is not a new topic and has been addressed since the early stages of its development . CI4CB has been involved in this long-lasting effort with the creation of fuzzy logic models capable of making accurate predictions while providing a reasonable level of interpretability to human users. Fuzzy logic allows for the creation of linguistic variables that may be semantically close to human language. These variables are then assembled in the form of IF-THEN rules that reflect the behavior of the modeled system. CI4CB work focused primarily on developing a modeling approach able to automatically create and test its own set of rules by means of an evolutionary algorithm. This work in particular, after several successful application projects in medical diagnosis, led to the creation of SimplicityBio, a spinoff from our group that provides clients with services for discovering biomarkers and interpretable diagnostic signatures, based on this methodology.
From 2016 to 2018, under a Hasler foundation grant, our group has investigated rule extraction methods from deep  neural networks. The goal of this project was to study, implement, and evaluate methods to better understand how deep neural networks make their predictions. Since its beginning, the project has taken a special focus on convolutional neural networks (CNNs) used primarily for image analysis.

Research projects

GRANTS AND EXTERNAL FUNDING

  • D-Rex: Deep Rule Extraction from Deep Neural Networks. 2016-2018. National funding: Hasler Foundation
  • BOSS Explorer: Development and integration of new methods for exploration, selection, and visualization of diagnostic signatures. 2016-2017. National Funding: CTI Agency. In collaboration with SimplicityBio SA, Monthey, Switzerland.
  • nanoFUGE: FUzzy modelling Gene-Expression data from Nanostring technology. 2010-2014. National Funding: SNSF Project. In collaboration with the University of Geneva and the University Hospitals of Geneva. In the frame of the SNSF Project “Leukaemia diagnosis using Nanotechnology”.
  • Smart Building. Système de gestion énergétique des bâtiments basé sur des techniques d’intelligence artificielle (System for energetic management of buildings based on artificial intelligence). 2010-2012. National funding: RCSO. In collaboration with Fribourg University of Applied Sciences.
  • NavMedia: Navigation and exploration of self-organized business media. 2008-2009. National Funding: CTI Agency. In collaboration with the Ipseité SA, Ecublens, Switzerland.
  • MyDeskPerso: Moteur de personnalité artificielle pour le robot-jouet Mydeskfriend. 2008-2009. National Funding: CTI Agency. In collaboration with Arimaz SA, Renens, Switzerland.
  • FASTPricer: FPGA-based Accelerating Solution for Parallel Synthetic CDO Pricer. 2008-2009. National Funding: CTI Agency. In collaboration with Risk Metrics Group and IOxOS Technologies, Switzerland.

PATENTS

Related Publications

Peer reviewed conferences

  1. Z. Mungloo-Dilmohamud, G. Marigliano, Y. Jaufeerally-Fakim, and C.A. Peña-Reyes, “A Comparative Study of Feature Selection Methods for Biomarker Discovery”, Submitted for peer reviewing to the 2018 IEEE International Conference on Bioinformatics and Biomedicine (IEEE BIBM 2018).
  2. D-V. Ramírez López, C.A. Peña-Reyes, and Á-J. Rojas, “Agent-based modeling of mesenchymal stem cells on a 3D-printed bio-device for the regenerative treatment of the infarcted myocardium”, Submitted for peer reviewing to the 2018 IEEE International Conference on Bioinformatics and Biomedicine (IEEE BIBM 2018).
  3. J. Despraz, S. Gomez, H.F. Satizábal and, C.A. Peña-Reyes, Towards a Better Understanding of Deep Neural Networks Representations using Deep Generative Networks, IJCCI 2017. SCITEPRESS 215-222, 2017. DOI
  4. M.A. Barreto-Sanz, A. Bujard and C.A. Peña-Reyes, Evolving very compact fuzzy systems for gene expression data analysis. Proceedings of the IEEE 12th International Conference on BioInformatics and BioEngineering, BIBE 2012, pp. 356-361, 2012.
    DOI – PDF
  5. J. Rossier, C.A. Peña-Reyes, Extrinsic evolution of fuzzy systems applied to disease diagnosis, ICES 2010. Lecture Notes in Computer Science, Vol. 6274/2010, 226-237, 2010. DOI
  6. M.A. Barreto-Sanz, A. Perez-Uribe, C. A. Peña-Reyes, M. Tomassini, Fuzzy Growing Hierarchical Self-Organizing Networks. ICANN 2008, LNCS 5164, pp. 713-722, 2008. DOI
  7. M.A. Melgarejo, C.A. Peña-Reyes, E. Sanchez. A genetic-fuzzy system approach to control a model of the HIV infection dynamics. Proceedings of the IEEE International Conference on Fuzzy Systems, pp. 2323 – 2330, 2006. DOI – PDF
  8. E. Jodoin, C.A. Pena-Reyes, E. Sanchez. A method for the fuzzification of categorical variables. Proceedings of the IEEE International Conference on Fuzzy Systems, 2006, pp.831 – 838, 2006. DOI – PDF
  9. C.A. Peña-Reyes. Incremental fuzzy coco: Incremental coevolution of fuzzy systems. Proceedings of the International Congress on Computational Intelligence – ICCI 2003. RedCIC – Colombian Network on Computational Intelligence, 2003.
  10. C.A. Peña-Reyes. Island fuzzy coco: Island model-based coevolution of fuzzy systems. Proceedings of the Third European Symposium on Intelligent Technologies, Hybrid Systems and their implementation on Smart Adaptive Systems – EUNITE 2003, pages 353-362. ELITE Foundation, Verlag Mainz, 2003.
  11. C.A. Peña-Reyes, R. Villa, L. Prieto, and E. Sanchez. COBRA: an evolved online tool for mammography interpretation. J. Mira and J. R. _Alvarez, editors, Proceedings of the International Work-conference on Artificial and Natural Neural Networks – IWANN2003, number 2686 in Lecture Notes in Computer Science, pages 726-733. Springer, 2003. DOI
  12. C.A. Peña-Reyes, M. Sipper, and L. Prieto. Sensitive, specific, and interpretable: Evolving a fuzzy mammographic-interpretation assessment tool. 2002 IEEE International Fuzzy Systems Conference Proceedings, pages 837-842. IEEE Neural Network Society, 2002. DOI – PDF
  13. C.A. Peña-Reyes and M. Sipper. The flowering of Fuzzy CoCo: Evolving fuzzy iris classifiers. V. Kurkova, N. C. Steele, R. Neruda, and M. Karny, editors, Proceedings of the 5th International Conference on Artificial Neural Networks and Genetic Algorithms (ICANNGA 2001), pages 304-307. Springer-Verlag, 2001. DOI
  14. C.A. Peña-Reyes and M. Sipper. Applying Fuzzy CoCo to breast cancer diagnosis. Proceedings of the 2000 Congress on Evolutionary Computation (CEC00), volume 2, pages 1168-1175. IEEE Press, Piscataway, NJ, USA, 2000. DOI – PDF
  15. C.A. Peña-Reyes and M. Sipper. Designing breast cancer diagnostic systems via a hybrid fuzzy-genetic methodology.
    IEEE International Fuzzy Systems Conference Proceedings, volume 1, pages 135-139. IEEE Neural Network Council, 1999. DOI – PDF
  16. C.A. Peña-Reyes and M. Sipper. Evolving fuzzy rules for breast cancer diagnosis. Proceedings of 1998 International Symposium on Nonlinear Theory and Applications (NOLTA’98), volume 2, pages 369-372. Presses Polytechniques et Universitaires Romandes, Lausanne, 1998.

Journal Papers

  1. C.A. Peña-Reyes and M. Sipper. Fuzzy CoCo: A cooperative-coevolutionary approach to fuzzy modelling. IEEE Transactions on Fuzzy Systems, 9(5):727-737, October 2001. DOI – PDF
  2. C.A. Peña-Reyes and M. Sipper. A fuzzy-genetic approach to breast cancer diagnosis. Artificial Intelligence in Medicine, 17(2):131-155, October 1999.
    DOI – PDF

Book chapters

  1. M.A. Barreto-Sanz, Andres Perez-Uribe, Carlos-Andres Pena-Reyes and Marco Tomassini. Tuning parameters in Fuzzy Growing Hierarchical Self-Organizing Artificial Neural Networks.
    Series: Studies in Computational Intelligence , Vol. 258 In. Franco, Leonardo; Elizondo, David A.; Jerez, José M. (Eds.) Constructive Neural Networks. 2009.
    DOI
  2. C.A. Peña-Reyes and M. Sipper. Fuzzy CoCo: Balancing accuracy and interpretability of fuzzy models by means of coevolution.
    J. Casillas, O. Cordón, F. Herrera, and L. Magdalena, editors, Accuracy Improvements in Linguistic Fuzzy Modelling, volume 129 of Studies in Fuzziness and Soft Computing, pages 119-146. Physica-Verlag, 2003. DOI
  3. C.A. Peña-Reyes and M. Sipper. Combining evolutionary and fuzzy techniques in medical diagnosis.
    M. Schmitt, H. N. Teodorescu, A. Jain, A. Jain, S. Jain, and L. C. Jain, editors, Computational Intelligence Techniques in Medical Diagnosis and Prognosis, volume 96 of Studies in Fuzziness and Soft Computing, chapter 14, pages 391-426. Springer-Verlag, Heidelberg, 2002. DOI

Book

  • C.A. Peña-Reyes. Coevolutionary Fuzzy Modelling.
    Lecture Notes in Computer Science 3204. Springer, 2004. 129 pages. ISBN: 3540229949. DOI

Ph.D. Thesis

  • C.A. Peña-Reyes. Coevolutionary Fuzzy Modelling.
    PhD thesis, École Polytechnique Fédérale de Lausanne – EPFL, 2002. Nominee, 2002 Best thesis EPFL prize. PDF

Invited lectures&talks

  1. Introduction to machine learning methods for data analysis. University « Autonoma de Occidente », Cali, Colombia, May 2015.
  2. Introduction à la programmation de robots pour gymnasiens (Introduction to robot programming for high-school students). HEIG-VD. 2012-2015
  3. Modélisation mathématique et informatique dans le génie médical. Cours DDM (HEIG-VD). 2009-2015
  4. Evolutionary Computing, Fuzzy Logic, Fuzzy Modeling, and Evolutionary Fuzzy Modeling, Invited lectures, Prof. Gianluca Tempesti’s course, EPFL, Lausanne, Switzerland, June 2005 to February 2006.

Media

Tools

Posters

Gallery

Student projects supervised

Co-Supervision of PHD Thesis

  • Smart Adaptive Artifacts, Andrés Upegui, EPFL, 2003-2006
  • Knowledge extraction from multi-layer perceptrons for modeling linear dynamic systems, Jesús Alfonso López-Sotelo, Universidad del Valle, Cali, Colombia, 2003-2007

Master Theses and Bachelor projects

  • Ensembles de classificateurs pour la sélection de variables diagnostiques avec microarrays (Ensembles of classifiers for diagnostic feature selection on microarray data), Gary Marigliano, HES-SO, 2016
  • Neuro-Evolutionary Modeling for Feature Selection. Kevin Jaquier, HEIG-VD, 2014
  • Integrating Categorical Variables in Fuzzy Modelling for Biomarker Selection. Diogo Leite. HEIG-VD and ISEP (Portugal), 2014. In collaboration with SimplicityBio, Monthey, Switzerland.
  • Intelligence artificielle, mémoire et courbes de l’oubli (Artificial intelligence, memory and the forgetting curve) , Pierre-Yves Gerber, HEIG-VD, 2013
  • Jeux et Intelligence Artificielle: Personnages artificiels à logique floue (Games and Artificial Intelligence: Fuzzy logic-based artificial characters), Michaël Glauser, HEIG-VD, 2013
  • Couplage apprentissage – évolution pour la modélisation floue (Coupling learning and evolution for fuzzy model builing), Yvan Da Silva, HEIG-VD, 2012
  • Jeux et Intelligence Artificielle: Conducteur de voiture de course à logique floue (Games and artificial intelligence : Fuzzy logic-based race car driver), Julien Schaller, HEIG-VD, 2012
  • Librairie graphique dédiée à l’interprétation de modèles bio-inspirés (Graphic library for interpreting bio-inspired models), Pascal Berberat, HEIG-VD, 2011
  • ODE-like fuzzy modeling of biological networks, Stephane Schnyder, EPFL-Novartis, 2007
  • Fuzzy modeling involving categorical variables, Etienne Jodoin, EPFL-Novartis, 2006
  • Evolutionary fuzzy modeling of biological networks, Thibaut Christe, EPFL-Novartis, 2006
  • Learning strategy for adaptive interpretable fuzzy models, Luc Dobler, EPFL, 2004
  • Methodology for conceiving fuzzy logic-based sequential machines, Gaétan Conti, EPFL, 2004
  • Incremental Fuzzy CoCo: further investigations, Etienne Jodoin, EPFL, 2004
  • Implementation and application of evolutionary fuzzy sequential machines, Lucien Epiney, EPFL, 2004
  • Hardware implementation of type-2 fuzzy logic systems, Miguel Alberto Melgarejo-Rey, Universidad de los Andes, Bogotá, Colombia, 2003
  • Optimization of an automotive suspension using genetic algorithms, Nicolas Dekker and Cyrill Fréling, EPFL, 2003
  • Parallel genetic algorithms: minimization of Moiré for serigraphic printing, Nicolas Frey, EPFL, 2003
  • Time-guided spiking neural networks, Raúl Camacho, EPFL, 2003
  • Fuzzy modeling with Particle Swarm Optimization, Julian Barbier and Jo Appenzeller, EPFL, 2003
  • Incremental Fuzzy CoCo, Olivier Rutti, EPFL, 2002.
  • Fuzzy Modeling by (r + 1)-Fuzzy CoCo, Michael Bruderer, EPFL, 2002.
  • Test of the applicability of Fuzzy CoCo to function approximation, Akhil Gupta, Indian Institute of Technology, Kanpur (Summer Internship Program, EPFL), 2001.
  • Island model for the cooperative coevolutionary algorithm Fuzzy CoCo, Yves Blatter, EPFL, 2001.
  • Real-time evolution of a neuro-fuzzy robot controller, Rodolph Perfetta, EPFL, 2000.
  • A study of the Fuzzy CoCo algorithm, Jean-Marc Henry, EPFL, 2000.
  • Evolution of the adaptability of a neuro-fuzzy controlled autonomous robot, Didier Rizzotti, EPFL, 2000.
  • A reconfigurable neuro-fuzzy processor using on-line arithmetic, Alvaro Gutiérrez (in cooperation with J.-L. Beuchat), EPFL, 2000.
  • Intelligent data analysis, Francesc Font (in cooperation with J.-L. Beuchat), EPFL, 2000
  • Hybrid learning (neural-evolutionary) of a fuzzy system for breast cancer diagnosis, Jean-Francois Philagor (in cooperation with M. Sipper), EPFL, 1999.
  • A reconfigurable digital fuzzy coprocessor, Christophe Gogniat (in cooperation with A. Pérez-Uribe), EPFL, 1999.