Towards Attention-Based ANN Models for Tabular Data Regression: A Use Case of Water Desalination

Abstract

In this paper, we propose a novel machine learning model applied to water desalination processes as a potential use case. Machine learning models show great promise in diverse fields, unlike model-based, often complex mathematical tools, due to their model-free nature and ability to learn from new data. In particular, we start with a classic artificial neural network (ANN) model and explore the potential of boosting its prediction performance using recently introduced attention-based models. This is among the early work to explore attention models for tabular data regression. Moreover, to the best of the authors' knowledge, it is the first to apply attention-based ANN models for tabular data regression to air gap membrane distillation (AGMD) water desalination processes. To test the merits of our proposed approach and better understand potential tradeoffs, we rely on a public dataset representing AGMD water desalination processes to predict key system parameters. We demonstrate the superior performance of our proposed model compared to a classic ANN-based baseline model in [1], which is attributed primarily to incorporating attention.