Modeling of complex biological systems, especially at a molecular scale, is an emerging field of research, inspired by the recent development of high throughput techniques in molecular biology. The corresponding objective for mathematical modeling is to be able to analyze the behavior of these high dimensional dynamical systems. This is an important challenge, because the understanding of normal and pathological functioning of cells at a molecular level could guide us to develop targeted therapies for systemic diseases such as cancer. To free ourselves from problems related to parameter uncertainty, this thesis propose to work with orders of magnitude, instead of accurate parameter values. This leads us naturally to the use of tropical analysis for obtaining reduced and hybrid models. These developments open new mathematical perspectives regarding dynamical systems. We obtain some results concerning the comparison between the solutions of differential equations systems and the solutions of truncated, piecewise smooth systems obtained by tropicalization. Another part of this work is dedicated to the numerical study of hybrid systems. The question in this part is how to build a hybrid model which reproduces a given experimental behavior, and how to identify the parameters of the hybrid model from time-series data. We propose an original identification algorithm, combining linear and nonlinear programming. This algorithm splits the problem into two subproblems : the identification of mode behavior parameters that is solved by simulated annealing, and the identification of mode control parameters, that is solved by linear programming. Relatively large scale applications are addressed by this approach, notably a mammalian cell cycle model.