Agricultural intensification is leading to a severe decline in farmland biodiversity worldwide. The resulting landscape simplification through the expansion of monocultures and removal of non-crop habitats has a major impact on arthropod communities in agricultural landscapes. While arable fields are often highly disturbed and ephemeral habitats that are unsuitable for many species, non-crop habitats in agroecosystems can provide important refugia. The creation of non-crop habitats through agri-environmental schemes (AES) in intensive agricultural landscapes, such as the ‘Maifeld’ region in western Germany, is intended to mitigate the negative effects of agricultural intensification, although the effectiveness of these measures for nature conservation is still controversial. Therefore, this work focuses on the taxonomic and functional diversity of beetles (Coleoptera) and spiders (Araneida), being important providers of ecosystem services, between wheat fields and different non-crop habitats, namely grassy field margins adjacent to wheat and oilseed rape fields, small- and large-scale set-aside areas sown with wildflowers, and permanent grassland fallows. Arthropods were collected between 2019 and 2020 using pitfall traps and suction sampling. Land-use type influenced beetle and spider diversity in the study area, with significantly higher values in grassland fallows than wheat fields. Surprisingly, species diversity differed little among all non-crop habitats, but all harboured distinct species assemblages. In particular, large long-term grassland fallows showed the largest within-group variation of beetle and spider assemblages and represented important habitats, especially for habitat specialists and threatened species, likely due to their variable soil moisture and complex habitat structure. In contrast, the homogeneous arthropod assemblages of wheat fields exhibited lower trait richness and were dominated by a few predatory species adapted to such disturbed, man-made habitats. Interestingly, all conservation measures complemented each other in that they contributed in different ways to supporting beetles and spiders in agricultural landscapes. Even small-scale non-crop habitats and existing habitat boundaries in an agricultural matrix appear to be valuable habitats for farmland arthropods by enhancing taxonomic diversity. Field margins and small wildflower-sown patches can link isolated non-crop habitats and contribute to a heterogeneous agricultural landscape. Consequently, a combination of various small- and large-scale greening measures leads to increased compositional and configurational landscape heterogeneity, resulting in improved beetle and spider diversity. Considering the ongoing loss of farmland biodiversity worldwide, agri-environmental schemes should be promoted in the future, as they are particularly important for arthropod conservation in intensive agricultural landscapes such as the Maifeld region.