Detecting signs of cognitive impairment as early as possible is one of the most urgent challenges in preventive care of dementia. It has still been unclear whether physical fitness measures can serve as markers of low cognitive function, a sign of cognitive impairment, in older people free from dementia. The aim of the present study was to examine an association between each of five physical fitness measures and global cognition in Japanese community-dwelling older adults without apparent cognitive problems. The baseline research of the Sasaguri Genkimon Study was conducted from May to August 2011 in Sasaguri town, Fukuoka, Japan. Of the 2,629 baseline subjects who were aged 65 years or older and not certified as individuals requiring nursing care by the town, 1,552 participants without apparent cognitive problems (Mini-Mental State Examination score ≥24) were involved in the present study (59.0% of the baseline subjects, median age: 72 years, men: 40.1%). Global cognitive function was measured by the Japanese version of the Montreal Cognitive Assessment. Handgrip strength, leg strength, sit-to-stand rate, gait speed, and one-leg stand time were examined as physical fitness measures. In multiple linear regression analyses, each of the five physical fitness measures was positively associated with the Montreal Cognitive Assessment score after adjusting for age and sex (p < 0.001). These associations were preserved after additional adjustment for years of formal education, body mass index, and other confounding factors (p < 0.001). The present study first demonstrated the associations between multiple aspects of physical fitness and global cognitive function in Japanese community-dwelling older people without apparent cognitive problems. These results suggest that each of the physical fitness measures has a potential as a single marker of low cognitive function in older populations free from dementia and thereby can be useful in community-based preventive care of dementia.