This paper proposes a high-order differential feedback controller (HODFC) and a fractional high-order differential feedback controller (FHODFC) to improve regulating ability of a commonly used automatic voltage regulator (AVR) system. In controller design process, particle swarm optimization (PSO) algorithm is utilized together with analytic approach. A constrained optimization problem is solved by PSO algorithm considering a specified objective function to obtain a less setting time, percentage overshoot, and regulation error. In order to test the performance of the proposed controllers, optimally tuned (proportional-integral-derivative) PID controllers available in the literature are implemented. The results demonstrate that the proposed FHODFC provides less percentage overshoot, settling time, rise time, and peak time than other proposed controllers, i.e., HODFC. Furthermore, the performance of the several available PID controllers is significantly worse than both of the proposed controllers in terms of transient response characteristics.