Although recent speech AI advances have enabled real-time speech understanding and generation, real-time accent conversion (AC) for second-language (L2) speakers in real-world interactions remains underexplored. In this paper, we present AccentDrift, a real-time streaming AC system that preserves linguistic content and speaker identity while transforming speech into a target accent with minimal latency. From an information-theoretic perspective, we adopt sparse speech tokenization to extract linguistic information from continuous speech. An accent adapter injects accent style into sparse semantic tokens, followed by a timbre adapter that hierarchically generates speaker-specific acoustics. To support streaming, we adopt a cache-aware streaming architecture and parallel streams, achieving 520 ms latency. AccentDrift outperforms prior parallel AC models, and demonstrates the feasibility of generating native-like accents for L2 speakers.