[Objective] With China’s ongoing electricity market reforms and the pursuit of carbon peaking and neutrality goals， renewable energy penetration in the power system is rapidly increasing. While supporting clean energy transition， this also introduces marked electricity price volatility and market uncertainty， highly complicating the development of bidding strategies by power producers， relying in particular on spot trading. In response to the development of the optimal bidding strategies by traditional thermal power enterprises and diverse energy market players in the joint energy and frequency regulation ancillary services market， a bidding strategy optimization method based on proximal policy optimization （PPO） is proposed. [Methods] First， a bi-level optimization model is established for the joint energy-frequency regulation market， integrating multiple generation types and renewable energy storage， where storage smooths price fluctuations through charge-discharge control， elevating the risk response capability of market players such as wind-storage unions. In this framework， the upper-level power producers develop bidding strategies aiming at profit maximization， while the lower-level market clearing model achieves joint dispatch with the objective of minimizing system operating costs. Second， the bidding problem is formulated as a Markov decision process （MDP） within a deep reinforcement learning （DRL） framework， where PPO algorithm is employed to achieve autonomous learning and dynamic optimization of bidding strategies. [Results] Comparative analysis against the theoretical optimal solution in typical cases demonstrates that the proposed approach effectively boosts thermal power enterprises’ revenues， mitigates the risks resulting from renewable energy price fluctuations， reduces system operating costs， and enhances frequency regulation efficiency. [Conclusions] The proposed approach demonstrates superior economic performance and higher real-time computational efficiency in a joint market compared with benchmark solutions.

[Objective] In the context of large-scale grid integration of renewable energy and deepening reforms in the electricity market， a new collaborative optimization framework consisting of multi-virtual power plants （MVPP） and an aggregate-level virtual power plant operator （VPPO） is established to address the challenges posed by the uncertainty of wind and solar power output， the complexity of electricity-hydrogen energy complementarity， and multi-level market coordination. [Methods] First， the Wasserstein distance is employed to construct a fuzzy set for wind and solar power output， and a two-stage distribution robust optimization model is combined to quantify the risk of prediction errors. Subsequently， a Stackelberg-Nash bi-level game framework is established， where VPPO， as the leader， dynamically formulates strategies for electricity/hydrogen energy trading prices and resource allocation， while MVPP， as the follower， optimizes electricity purchase and sale plans， hydrogen scheduling， and flexible load response. Finally， the multi-agent twin delayed deep deterministic policy gradient （MATD3） algorithm is adopted to efficiently solve high-dimensional non-convex problems through the utilization of a centralized training and decentralized execution mechanism and a dual Critic network to mitigate strategy bias. [Results] The numerical results demonstrate that the proposed strategy successfully optimizes the cross-market arbitrage profits of VPPO and the operational costs of MVPP， significantly enhancing the convergence efficiency and strategy stability of the algorithm. It also bolsters the system's robustness against fluctuations in wind and solar power and the flexibility of electricity-hydrogen coordination， while optimizing the spatiotemporal distribution of resources through flexible peak load shaving services. [Conclusions] By integrating robust optimization and game theory， this study effectively coordinates electricity-hydrogen coupling with multi-market interactions， providing an innovative solution to the economic and robust operation of MVPP under high penetration of integrated renewable energy.