Knowledge is one of the fundamental cognitive objects in the brain among those of data, information, and intelligence. Knowledge can be classified into two main categories, ie, conceptual knowledge for knowing to-be and behavioral knowledge for knowing to-do, particularly the former. This paper presents a basic study on a mathematical theory of knowledge towards knowledge science. The taxonomy and cognitive foundations of knowledge are explored, which reveal that the basic cognitive structure of conceptual knowledge is a formal concept and that of behavioral knowledge is a formal process. Mathematical models of knowledge are created in order to enable formal representation and rigorous manipulation of knowledge. A set of formal principles and properties of knowledge is elicited and elaborated towards the development of knowledge science and cognitive knowledge systems. It is discovered that the basic unit of knowledge is a binary relation, shortly bir, as a counterpart of bit (a binary digit) for information and data.