AI Literacy and Perceived Reliability as Predictors of Student Trust in Generative AI-Assisted Decision Making

Ika Safitri Windiarti; Agung Prabowo; Fathimah Azzahra

doi:10.33506/insect.v12i2.6235

Authors

Ika Safitri Windiarti Program Studi Ilmu Komputer, Universitas Muhammadiyah Palangkaraya
Agung Prabowo Program Studi Sistem Informasi, STMIK Palangkaraya
Fathimah Azzahra Bachelor of Science (Multimedia), Universiti Utara Malaysia

DOI:

https://doi.org/10.33506/insect.v12i2.6235

Keywords:

AI literacy, perceived reliability, user trust, generative AI, decision support, university students

Abstract

Generative artificial intelligence is increasingly used by university students to compare alternatives, obtain recommendations, and support academic decisions. This study examines whether objective AI literacy and perceived reliability can predict student trust in generative AI-assisted decision-making. A cross-sectional online survey yielded 406 responses. After eligibility screening, 310 respondents met the inclusion criteria, and 280 respondents who passed the attention check served as the primary analysis sample. AI literacy was measured using 12 multiple-choice questions, while perceived reliability and user trust were measured on a five-point scale. Item analysis supported the removal of two negatively worded items that had low item-total correlations. Multiple linear regression with HC3 heteroskedasticity-robust standard errors showed that the model explained 52.1% of the variance in user trust. Perceived reliability was the dominant predictor (standardized beta = 0.704; p < 0.001), whereas AI literacy provided a smaller positive contribution (standardized beta = 0.101; p = 0.022). The AI literacy coefficient was non-significant in a sensitivity analysis that retained all eligible respondents. These findings indicate that student trust primarily depends on the assessment that generative AI outputs are accurate, consistent, relevant, and reliable. Higher education institutions need to integrate AI literacy education with verification practices so that students are capable of assessing system reliability before using AI recommendations.

References

[1] A. A. N. Gunawan, I. S. Windiart, U. B. Nasutio, S. T. Utomo, and S. Asuhadi, “Investigating the role of digital transformation, digital knowledge management and digital innovation on sustainable performance (SDG) of SMES,” Digital Knowledge Management And Digital Innovation On Sustainable Performance (SDG) Of SMES (September 24, 2024), 2024.

[2] T. H. I. Alam and I. S. Windiarti, “The future of artificial intelligence in interactive learning: Trends, challenges, opportunities,” Engineering Proceedings, vol. 84, no. 1, p. 87, 2025.

[3] M. Miftahurrisqi, I. S. Windiarti, and D. Haryanto, “Edukasi dan Peningkatan Kompetensi It-Security Dan E-Commerce Bagi Mahasiswa Ma’had Asy-Syafi’i Universitas Muhammadiyah Palangkaraya,” Jurnal Martabe Universitas Muhammadiyah Tapanuli Selatan, vol. 4, no. 3, pp. 963–969, 2021.

[4] I. Windiarti, “Implementasi Website Sebagai Media Informasi pada Panti Asuhan Budi Mulya Palangkaraya,” Jurnal Pengabdian Dharma Laksana.

[5] C. Baek, T. Tate, and M. Warschauer, “ChatGPT seems too good to be true: College students’ use and perceptions of generative AI,” Computers and Education: Artificial Intelligence, vol. 7, p. 100294, 2024, doi: 10.1016/j.caeai.2024.100294.

[6] J. D. Lee and K. A. See, “Trust in automation: Designing for appropriate reliance,” Hum. Factors, vol. 46, no. 1, pp. 50–80, 2004, doi: 10.1518/hfes.46.1.50_30392.

[7] M. T. Dzindolet, S. A. Peterson, R. A. Pomranky, L. G. Pierce, and H. P. Beck, “The role of trust in automation reliance,” Int. J. Hum. Comput. Stud., vol. 58, no. 6, pp. 697–718, 2003, doi: 10.1016/S1071-5819(03)00038-7.

[8] A. D. Kaplan, T. T. Kessler, J. C. Brill, and P. A. Hancock, “Trust in artificial intelligence: Meta-analytic findings,” Hum. Factors, vol. 65, no. 2, pp. 337–359, 2023, doi: 10.1177/00187208211013988.

[9] M. F. Shahzad, S. Xu, and I. Javed, “ChatGPT awareness, acceptance, and adoption in higher education: The role of trust as a cornerstone,” International Journal of Educational Technology in Higher Education, vol. 21, p. 46, 2024, doi: 10.1186/s41239-024-00478-x.

[10] K. A. Mariñas, C. S. Saflor, P. Alvarado, J. M. Uminga, and N. A. Verde, “Assessing the importance of variables from a revised technology acceptance model for the use of ChatGPT by university students,” Computers and Education: Artificial Intelligence, vol. 9, p. 100435, 2025, doi: 10.1016/j.caeai.2025.100435.

[11] B. Foroughi, M. Iranmanesh, M. Ghobakhloo, and M. G. Senali, “Determinants of ChatGPT adoption among students in higher education: The moderating effect of trust,” The Electronic Library, vol. 43, no. 1, pp. 1–21, 2025, doi: 10.1108/EL-12-2023-0293.

[12] A. Polyportis and N. Pahos, “Understanding students’ adoption of the ChatGPT chatbot in higher education: The role of anthropomorphism, trust, design novelty and institutional policy,” Behaviour & Information Technology, vol. 44, no. 2, pp. 315–336, 2025, doi: 10.1080/0144929X.2024.2317364.

[13] D. Long and B. Magerko, “What is AI literacy? Competencies and design considerations,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1–16. doi: 10.1145/3313831.3376727.

[14] D. T. K. Ng, J. K. L. Leung, S. K. W. Chu, and M. S. Qiao, “Conceptualizing AI literacy: An exploratory review,” Computers and Education: Artificial Intelligence, vol. 2, p. 100041, 2021, doi: 10.1016/j.caeai.2021.100041.

[15] I. S. Windiarti, S. Bahri, and A. Prabowo, “Melangkah maju dengan teknologi generative AI: Peningkatan kompetensi kepala sekolah SMP di Kota Palangkaraya,” Parta: Jurnal Pengabdian Kepada Masyarakat, vol. 4, no. 1, pp. 46–52, 2023.

[16] M. Hornberger, A. Bewersdorff, and C. Nerdel, “What do university students know about artificial intelligence? Development and validation of an AI literacy test,” Computers and Education: Artificial Intelligence, vol. 5, p. 100165, 2023, doi: 10.1016/j.caeai.2023.100165.

[17] R. Salhab and M. M. Aboushi, “Influence of AI literacy and 21st-century skills on the acceptance of generative artificial intelligence among college students,” Front. Educ. (Lausanne)., vol. 10, p. 1640212, 2025, doi: 10.3389/feduc.2025.1640212.

[18] K. E. Schaefer, J. Y. C. Chen, J. L. Szalma, and P. A. Hancock, “A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems,” Hum. Factors, vol. 58, no. 3, pp. 377–400, 2016, doi: 10.1177/0018720816634228.

[19] M. C. Laupichler, A. Aster, and T. Raupach, “Delphi study for the development and preliminary validation of an item set for the assessment of non-experts’ AI literacy,” Computers and Education: Artificial Intelligence, vol. 4, p. 100126, 2023, doi: 10.1016/j.caeai.2023.100126.

[20] I. S. Windiarti, “Kajian Literatur Trend Penelitian Di Bidang Informatika Dan Komputer Untuk Dosen Dan Mahasiswa,” Jurnal Sains Komputer dan Teknologi Informasi (JSAKTI), vol. 3, no. 2, pp. 114–118, 2021.

[21] C. Yelia, I. S. Windiarti, M. Miftahurrizqi, and A. Prabowo, “Aplikasi Pendaftaran Darul Arqom Dasar (Dad) Pada Ikatan Mahasiswa Muhammadiyah (Imm) Averroes Palangka Raya Berbasis Web Responsive: Web-Responsive Application for Darul Arqom Dasar Registration (DAD) in the Muhammadiyah Students Association (IMM) Averroes Palangka Raya,” Jurnal Sains Komputer dan Teknologi Informasi, vol. 7, no. 1, pp. 66–71, 2024.

[22] I. Amri and I. S. Windiarti, “Information Technology Challenges: Developments and Potential Impact on Socio Economics in the Next Two Decades,” International Journal of Engineering, Science and Information Technology, vol. 5, no. 4, pp. 1–8, Jul. 2025, doi: 10.52088/ijesty.v5i4.917.

[23] I. S. Windiarti and M. Miftahurrizqi, “Perencanaan Implementasi Komputasi Awan Pada Infrastruktur Teknologi dan Sistem Informasi di UMPR: Implementation Of Cloud Computing Planning in Technology and Information Systems Infrastructure at Muhammadiyah University of Palangkaraya,” Jurnal Sains Komputer dan Teknologi Informasi, vol. 4, no. 2, pp. 59–64, 2022.

[24] A. Prabowo, I. S. Windiarti, and A. R. Kurniawan, “Aplikasi Sistem Survei Persepsi Layanan Dan Persepsi Korupsi Pada Kantor Imigrasi Kelas I Non Tpi Palangka Raya Berbasis Framework Codeigniter: Application of Service Perception Survey System and Corruption Perception in Class I Immigration Office Non Tpi Palangka Raya Based on Codeigniter Framework,” Jurnal Sains Komputer dan Teknologi Informasi, vol. 3, no. 2, pp. 70–74, 2021.

[25] N. Rizun, O. N. Bordean, A. Nikiforova, I. N. Beleiu, and A. Revina, “Generative AI in higher education: Ethical and behavioral factors influencing students’ intentions to use ChatGPT,” Computers and Education Open, vol. 10, p. 100336, 2026, doi: 10.1016/j.caeo.2026.100336.