Knowledge regarding accelerometer-derived physical activity (PA) and sedentary behavior (SED) levels is scarce for Japanese older adults. The aims of this study were therefore to 1) describe levels of PA and SED in Japanese community-dwelling older adults, using tri-axial accelerometer; 2) examine the variation of PA and SED with respect to sex, age, and body mass index (BMI). Participants of this study were from the baseline survey of the Sasaguri Genkimon Study, who were 65 years or older and not certified as those requiring long-term care. PA was assessed objectively for seven consecutive days using tri-axial accelerometer. A total of 1,739 participants (median age: 72 years, men: 38.0%) with valid PA data were included. Overall, participants in the present study spent 54.5% of their waking time being sedentary and 45.5% being active, of which 5.4% was moderate-to-vigorous physical activity (MVPA). Women accumulated more minutes of light physical activity (LPA) and MVPA compared with men. In contrast, men spent more time being sedentary. Mean steps per day did not differ between sexes. Furthermore, participants with higher BMI (BMI ≥25) had lower PA levels, and longer SED compared with those with lower BMI (BMI <). PA levels were lower and SED was longer with age. The present study is the first to demonstrate that the levels of PA and SED differed by sex, age, and BMI in Japanese community-dwelling older adults. In particular, women were more active compared with men, providing unique insight into the current level of PA in older adults. Data presented in the study will enable further investigation of additional determinants of PA and SED in order to develop effective population-based intervention strategies to promote PA and reduce prolonged SED in the Japanese population and possibly other rapidly aging societies.