Adaptive cellular immunity plays a major role in clearing microbial invasion of mucosal tissues in mammals. Following the clearance of primary pathogens, memory lymphocytes are established both systemically and locally at pathogen entry sites. Recently, resident memory CD8 T and B cells (TRM and BRM respectively), which are parked mainly in non-lymphoid mucosal tissues, were characterized and demonstrated to be essential for protection against secondary microbial invasion. Here we reviewed the current understanding of the cellular and molecular cues regulating CD8 TRM and BRM development, maintenance and function. We focused particularly on elucidating the role of a novel tissue-resident helper T (TRH) cell population in assisting TRM and BRM responses in the respiratory mucosa following viral infection. Finally, we argue that the promotion of TRH responses by future mucosal vaccines would be key to the development of successful universal influenza or coronavirus vaccines, providing long-lasting immunity against a broad spectrum of viral strains.