RecA and its homologs help maintain genomic integrity through recombination. Using single-molecule fluorescence assays and hidden Markov modeling, we show the most direct evidence that a RecA filament grows and shrinks primarily one monomer at a time and only at the extremities. Both ends grow and shrink, contrary to expectation, but a higher binding rate at one end is responsible for directional filament growth. Quantitative rate determination also provides insights into how RecA might control DNA accessibility in vivo. We find that about five monomers are sufficient for filament nucleation. Although ordinarily single-stranded DNA binding protein (SSB) prevents filament nucleation, single RecA monomers can easily be added to an existing filament and displace SSB from DNA at the rate of filament extension. This supports the proposal for a passive role of RecA-loading machineries in SSB removal.