Hidden Markov model (HMM) has been well studied and extensively used. In this paper, we present DPHMM ({Differentially Private Hidden Markov Model}), an HMM embedded with a private data release mechanism, in which the privacy of the data is protected through a graph. Specifically, we treat every state in Markov model as a node, and use a graph to represent the privacy policy, in which "indistinguishability" between states is denoted by edges between nodes. Due to the temporal correlations in Markov model, we show that the graph may be reduced to a subgraph with disconnected nodes, which become unprotected and might be exposed. To detect such privacy risk, we define sensitivity hull and degree of protection based on the graph to capture the condition of information exposure. Then to tackle the detected exposure, we study how to build an optimal graph based on the existing graph. We also implement and evaluate the DPHMM on real-world datasets, showing that privacy and utility can be better tuned with customized policy graph.