第五代(5G)移动通信系统可望实现千倍的网络容量增长，已被列入国家重大战略部署。毫米波大规模MIMO技术因能显著提升通信带宽与频谱效率，被认为是5G最为核心的技术之一。然而，该技术需要大量射频链路，面临着高功耗高成本这一瓶颈。为此，本文深入研究了毫米波大规模MIMO中三种主流的低功耗结构，并针对这三种结构分别提出了性能优良的无线传输方案。具体贡献如下： 针对基于相控阵列的模拟结构，本文提出了基于禁忌搜索算法的多波束训练方案，以解决该结构下多波束训练开销过大这一问题。借鉴交替优化思想，本文首先提出采用多次单边训练的方式交替优化基站与用户的波束赋型矩阵。而每一次单边训练则结合人工智能中的禁忌搜索算法，利用邻域搜索与禁忌方向来高效搜索整个波束赋型码本。最后，本文还结合多次单边训练流程，在禁忌搜索算法中引入了随机重启机制，以进一步提升算法鲁棒性。理论分析与仿真结果表明，所提方案可在保证准最优传输速率前提下，降低训练开销约70％。 针对基于相控阵列的模数混合结构，本文提出了基于串行干扰消除思想的混合预编码方案，以解决该结构下尚无高效的子连接混合预编码这一问题。本文首先将具有非凸约束的混合预编码总速率优化问题分解为多个简单的子速率优化问题之和，其中每个子问题仅考虑一个子相控阵列。之后，借鉴串行干扰消除思想，我们逐一优化各子相控阵列，并在后续优化过程中消除其影响。最后，本文还提出了所提方案的一种低复杂度实现，以避免复杂的奇异值分解和矩阵求逆。仿真结果表明，所提方案具有准最优传输速率，并可提高能量效率2倍以上。 针对基于透镜阵列的模数混合结构，本文提出了基于连续支撑集检测的宽带波束信道估计方案，以解决该结构下波束扩散导致宽带波束信道估计精度损失严重这一问题。结合波束扩散现象的物理规律，本文首先证明宽带波束信道的路径分量具有特殊的频变稀疏结构。之后，本文提出将宽带波束信道估计问题分解为各路径分量估计的子问题。对于每个路径分量，本文利用上述频变稀疏结构来联合估计其在各个子载波上的支撑集以提高支撑集检测的准确度。理论分析与仿真结果表明，在相同的导频开销下，所提方案可显著提高宽带波束信道估计精度。 本文的相关研究成果可望克服低功耗毫米波大规模MIMO从理论走向应用所面临的挑战，为其在实际系统中的应用提供有力的技术支撑。 关键词：毫米波大规模MIMO；波束训练；混合预编码；波束信道估计

 The development of the fifth-generation (5G) wireless communications with significantly increased data rates has been recognized as a national major strategy. Millimeter-wave (㎜Wave) massive multiple-input multiple-output (MIMO), which can achieve much higher spectral efficiency and provide much wider bandwidth, has been considered as one of the most promising technology to realize the ambitious goal of 5G wireless communications. However, mmWave massive MIMO usually requires a large number of expensive radio frequency (RF) chains, resulting in unaffordable energy consumption and hardware cost. To solve this problem, in this thesis we investigate three promising energy-efficient architectures for mmWave massive MIMO, and propose three advanced transmission technologies for these architectures, respectively. The contributions of this thesis can be summarized as follows. For the phased array based analog architecture, the multiple-beam training usually involves quite high training overhead. To this end, we propose a tabu search (TS) based beam training scheme. Inspired by the idea of alternative optimization, we first adopt several times of single-side beam training to alternatively optimize the beamforming matrices at the base station and user sides. During each single-side beam training, we propose to use the TS algorithm developed from artificial intelligence, which utilizes the neighbor search and direction tabu, to search the large-size beamforming codebooks more efficiently. Moreover, combining with the procedure of several times of single-side beam training, we also propose a new random-restart mechanism for TS algorithm to further improve its robustness. The theoretical analysis and simulation results demonstrate that the proposed TS based beam training scheme can achieve the near-optimal achievable rate with more than 70% reduced training overhead. For the sub-phased array based hybrid (analog and digital) architecture, designing the efficient hybrid precoding is still an opening problem. To this end, we propose a successive interference cancelation (SIC) based hybrid precoding scheme. Inspired by the idea of SIC for multi-user signal detection, we first propose to decompose the total achievable rate optimization problem with non-convex constraints into a series of simple sub-rate optimization problems, each of which only considers one sub-phased array. Then, we prove that maximizing the sub-rate of each sub-phased array is equivalent to simply designing a feasible precoding vector sufficiently close (in terms of Euclidean distance) to the unconstrained optimal solution. Finally, we propose a low-complexity algorithm to realize the SIC based hybrid precoding, which can avoid the computation of singular value decomposition (SVD) and matrix inversion. Simulation results verify that the proposed SIC based hybrid precoding scheme is near-optimal, and enjoys more than twice the energy efficiency achieved by the conventional hybrid precoding schemes designed for the full-phased array based hybrid architecture. For the lens array based hybrid (analog and digital) architecture, the beam squint effect will make the existing wideband beamspace channel estimation schemes suffer from serious accuracy degradation. To this end, we propose a successive support detection (SSD) based beamspace channel estimation scheme. By exploiting the pattern of beam squint effect, we first prove that each path component of the wideband beamspace channel exhibits a unique frequency-dependent sparse structure. Inspired by this, we propose to successively estimate all the sparse path components. For each path component, its support at different sub-carriers is jointly estimated with high accuracy by utilizing the proved sparse structure, and its influence is removed to estimate the remaining path components. The theoretical analysis and simulation results show that compared with the existing wideband beamspace channel estimation schemes, the proposed SSD based beamspace channel estimation scheme can significantly improve the accuracy at the same pilot overhead. The results in this thesis are expected to overcome the challenges of ㎜ Wave massive MIMO with energy-efficient architectures, and promote its application in practice. Key words: Millimeter-wave massive MIMO; Beam training; Hybrid precoding; Beamspace channel estimation