究竟什么是java的'令牌'？

So I'm working on this program that opens all .java and .txt files in a specified directory, and does some analyzing on these files. One thing my program is supposed to find is the "Most frequently occurring token(s)" in each of the .txt and .java files. I only have a limited understanding of what qualifies as a token. Would a program that just finds the most common word in the file do the same thing? Or how do I go about specifying my method to find tokens, and not just words?So I'm working on this program that opens all .