In literature review, it is stated that students face a number of difficulties while they learn the concepts and the relationships between them in mathematics education, and that they often have difficulty in mathematics education. Therefore, mathematics educators have a consensus on developing problem-solving skills of students and that it should be the primary objective of the education. For this reason, the aim of this study is to assess the artificial intelligence based distance education system called ARTIMAT, designed to develop mathematical problem solving skills, in terms of the conceptual competence, the ease of use and students' contribution to the problem solving process. For this purpose, the application was tested with 59 students in 10th grade of an Anatolian High School in Trabzon. Firstly, every student was equipped to use the system individually and later, opinions of the students were taken through structured interviews. In the evaluation of the findings by the study, it was generally concluded that the system fulfilled the needs of the students and was successful. (C) 2014 Published by Elsevier Ltd.