Inferential Machine Learning: Towards Human-collaborative Foundation Models - presented by Professor Ghassan AlRegib

Inferential Machine Learning: Towards Human-collaborative Foundation Models

Professor Ghassan AlRegib

Professor Ghassan AlRegib
Ask the seminar a question! BETA
Inferential Machine Learning: Towards Human-collaborative Foundation Models
Professor Ghassan AlRegib
Ghassan AlRegib
Georgia Institute of Technology

Featured Link

G. AlRegib and M. Prabhushankar, “Explanatory Paradigms in Neural Networks: Towards Relevant and Contextual Explanations,” in IEEE Signal Processing Magazine, Special Issue on Explainability in Data Science, vol. 39, no. 4, pp. 59-72, Feb. 18, 2022.
Visit link

Neural network driven applications like ChatGPT suffer from hallucinations where they confidently provide inaccurate information. A funda- mental reason for this inaccuracy is the feed-forward nature of inductive decisions taken by neural networks. Such decisions are a result of training schemes that do not allow networks to deviate from and creatively abduce reasons at inference. With the advent of foundation models that are adapted across applications and data, humans can directly intervene and prompt vision-language foundation models. However, without understanding the operational limits of the underlying networks, human interventions often lead to unfair, inaccurate, hallucinated and unintelligible outputs. These outputs undermine the trust in foundation models, thereby causing roadblocks to their adoption in everyday lives. In this talk, we review systematic ways to analyze and understand human interventions in neural network functionality at inference. Specifically, we show that a human-AI collaborative environment via inferential machine learning techniques is a promising endeavor. We coined the term inferential machine learning in 2022 to reflect the opportunities and challenges in this space.

References
  • 1.
    G. AlRegib and M. Prabhushankar, “Explanatory Paradigms in Neural Networks: Towards Relevant and Contextual Explanations,” in IEEE Signal Processing Magazine, Special Issue on Explainability in Data Science, vol. 39, no. 4, pp. 59-72, Feb. 18, 2022.
  • 2.
    Ryan Benkert et al. (2024) Effective Data Selection for Seismic Interpretation through Disagreement.
  • 3.
    Mohit Prabhushankar and Ghassan AlRegib (2024) VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability.
  • 4.
    Kiran Kokilepersaud et al. (2024) HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms.
  • 5.
    Jorge Quesada et al. (2024) Benchmarking Human and Automated Prompting in the Segment Anything Model.
  • 6.
    M. Prabhushankar, and G. AlRegib, “VOICE: Variance of Induced Contrastive Explanations to Quantify Uncertainty in Neural Network Interpretability,” in Journal of Selected Topics in Signal Processing (J-STSP) Special Series on AI in Signal & Data Science, pp. 1-13, May. 23, 2024.
  • 7.
    J. Lee, C. Lehman, M. Prabhushankar, and G. AlRegib, “Probing the Purview of Neural Networks via Gradient Analysis,” in IEEE Access, vol. 11, pp. 32716-32732, Mar. 21, 2023.
  • 8.
    D. Temel, M. Prabhushankar, and G. AlRegib, “UNIQUE: Unsupervised Image Quality Estimation,” in IEEE Signal Processing Letters , vol. 23, no. 10, pp. 1414-1418, Oct., 2016.
  • 9.
    K. Kokilepersaud, S. Kim, M. Prabhushankar and G. AlRegib, “HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms,” in 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, AZ, Feb. 28 – Mar. 4, 2025. [ORAL (top 7.56%)]
  • 10.
    J. Quesada*, Z. Fowler*, M. Alotaibi, M. Prabhushankar, and G. AlRegib, “Benchmarking Human and Automated Prompting in the Segment Anything Model,” in IEEE Conference on Big Data 2024, Washington DC, USA, Dec. 15-18, 2024.
  • 11.
    R. Benkert, M. Prabhushankar, and G. AlRegib, “Targeting Negative Flips in Active Learning Using Validation Sets,” in IEEE Conference on Big Data 2024, Washington DC, USA, Dec. 15-18, 2024.
  • 12.
    K. Kokilepersaud, Yavuz Yarici, M. Prabhushankar, and G. AlRegib, “Taxes Are All You Need: Integration of Taxonomical Hierarchy Relationships Into the Contrastive Loss,” in IEEE International Conference on Image Processing (ICIP), Abu Dhabi, UAE, Oct. 27-30, 2024.
  • 13.
    R. Benkert, M. Prabhushankar, and G. AlRegib, “Transitional Uncertainty With Layered Intermediate Predictions,” in 41st International Conference on Machine Learning (ICML), Vienna, Austria, Jul. 21-27, 2024.
  • 14.
    M. Prabhushankar, and G. AlRegib, “Introspective Learning : A Two-Stage Approach for Inference in Neural Networks,” in Advances in Neural Information Processing Systems (NeurIPS), New Orleans, LA,, Nov. 29 – Dec. 1, 2022.
  • 15.
    R. Benkert, M. Prabhushankar, and G. AlRegib, “Forgetful Active Learning With Switch Events: Efficient Sampling for Out-of-Distribution Data,” in IEEE International Conference on Image Processing (ICIP), Bordeaux, France, Oct. 16-19, 2022.
  • 16.
    G. Kwon, M. Prabhushankar, D. Temel, and G. AlRegib, “Backpropagated Gradient Representations for Anomaly Detection,” in Proceedings of the European Conference on Computer Vision (ECCV), SEC, Glasgow, Aug. 23-28, 2020.
  • 17.
    G. Kwon*, M. Prabhushankar*, D. Temel, and G. AlRegib, “Distorted Representation Space Characterization Through Backpropagated Gradients,” in IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, Sep., 2019.
  • 18.
    D. Temel*, J. Lee*, and G. AlRegib, “CURE-OR: Challenging Unreal and Real Environments for Object Recognition,” in IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, Dec., 2018.
  • 19.
    D. Temel, G. Kwon*, M. Prabhushankar*, and G. AlRegib, “CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition,” in Advances in Neural Information Processing Systems (NIPS) Workshop on Machine Learning for Intelligent Transportation Systems, Long Beach, CA, Dec., 2017.
  • 20.
    DARai: Daily Activity Recordings for Artificial Intelligence and Machine learning
Research Communities by Springer Nature logo
Computational Science Seminars
Research Communities by Springer Nature
Cite as
G. AlRegib (2025, March 13), Inferential Machine Learning: Towards Human-collaborative Foundation Models
DOI
doi.org/10.52843/cassyni.z4r8dt
Export
Citations
Share
Timestamped permalink
Details
Listed seminar This seminar is open to all
Recorded Available to all
Video length 58:05
Q&A Now closed
Disclaimer The views expressed in this seminar are those of the speaker and not necessarily those of the journal