Modularity is a widespread feature of biological and artificial networks. It makes networks more easily evolvable, i.e. capable of rapidly adapting to new environments, and eventually more resilient. Despite its advantages, this characteristic is a controversial issue and the debate remains open on the nature of existing modules and the reasons for their emergence. In this work, we study modularity in evolutionary neural networks, i.e. artificial neural networks evolved to accomplish some predefined tasks. This evolution is performed by means of evolutionary optimization techniques such as genetic algorithms that mimic the Darwinian principles of natural evolution. With this aim, we develop simple applications in the frameworks of evolutionary robotics and pattern recognition. For each of them, we start our analysis by comparing the performance of modular and non-modular networks. We then focus on the search for modularity and we try to identify the conditions that cause its emergence by considering different hypotheses often met in the literature.