VSCode 黑魔法探秘之插件加载机制 作者: Semesse 时间: 2021-02-11 分类: 千叶 > 此处的 vscode 版本为 1.54.0,为还未发布的 master 分支,SHA 为 `afd102cbd2e17305a510701d7fd963ec2528e4ea` > 为了不让代码块太长,本文删掉了一些无关代码 最近一直想橄榄 vscode 的插件系统,让插件能够伪装成 extensionHost 调用 vscode 主进程中的内部服务,这样就可以不用对 vscode 源码进行魔改了,enableProposedApi 检测也可能绕过 首先我们可以找到 extensionHost 和 main 进行 rpc 的实现细节,大体上协议是 base64(rpcId + method + json(args))[1](#1) ```ts // src/vs/workbench/services/extensions/common/rpcProtocol.ts // 接受到消息查找 rpcId 对应的 actor 并调用对应方法 private _doInvokeHandler(rpcId: number, methodName: string, args: any[]): any { const actor = this._locals[rpcId]; if (!actor) { throw new Error('Unknown actor ' + getStringIdentifierForProxy(rpcId)); } let method = actor[methodName]; if (typeof method !== 'function') { throw new Error('Unknown method ' + methodName + ' on actor ' + getStringIdentifierForProxy(rpcId)); } return method.apply(actor, args); } // 调用 main 上的函数 private _remoteCall(rpcId: number, methodName: string, args: any[]): Promise { ... const serializedRequestArguments = MessageIO.serializeRequestArguments(args, this._uriReplacer); const req = ++this._lastMessageId; const callId = String(req); const result = new LazyPromise(); ... this._pendingRPCReplies[callId] = result; this._onWillSendRequest(req); const msg = MessageIO.serializeRequest(req, rpcId, methodName, serializedRequestArguments, !!cancellationToken); this._protocol.send(msg); return result; } ``` 麻烦的地方在于 rpcId 对应的 actor 是放在表中的(如下),`createXXXId` 调用的 `createProxyIdentifier` 每次都会 +1 作为新的 id,如果之后要跟随 vscode 更新的话肯定会有不一样的表[2](#2) ```typescript // src/vs/workbench/api/common/extHost.protocol.ts export const MainContext = { MainThreadAuthentication: createMainId('MainThreadAuthentication'), MainThreadBulkEdits: createMainId('MainThreadBulkEdits'), MainThreadClipboard: createMainId('MainThreadClipboard'), ... }; export const ExtHostContext = { ExtHostCommands: createExtId('ExtHostCommands'), ExtHostConfiguration: createExtId('ExtHostConfiguration'), ExtHostDiagnostics: createExtId('ExtHostDiagnostics'), ... }; ``` 那么插件有没有办法通过某种神秘方法拿到这个表呢?于是我们来看看插件是怎么加载的 ```ts // src/vs/workbench/api/common/extHost.api.impl.ts class Extension implements vscode.Extension { // extensionService 会在构造时注入 // 用了 private field 呕 #extensionService: IExtHostExtensionService; ... // 插件 activate 函数的返回值会作为 exports 供其他插件使用 get exports(): T { if (this.packageJSON.api === 'none') { return undefined!; // Strict nulloverride - Public api } return this.#extensionService.getExtensionExports(this.#identifier); } // 激活插件 activate(): Thenable { return this.#extensionService.activateByIdWithErrors(this.#identifier, { startup: false, extensionId: this.#originExtensionId, activationEvent: 'api' }).then(() => this.exports); } } // src/vs/workbench/api/common/extHostExtensionService.ts export abstract class AbstractExtHostExtensionService extends Disposable implements ExtHostExtensionServiceShape { ... // --- impl // impl of activation private async _activateExtension(extensionDescription: IExtensionDescription, reason: ExtensionActivationReason): Promise { if (!this._initData.remote.isRemote) { // local extension host process await this._mainThreadExtensionsProxy.$onWillActivateExtension(extensionDescription.identifier); } else { // remote extension host process // do not wait for renderer confirmation this._mainThreadExtensionsProxy.$onWillActivateExtension(extensionDescription.identifier); } return this._doActivateExtension(extensionDescription, reason).then((activatedExtension) => { const activationTimes = activatedExtension.activationTimes; this._mainThreadExtensionsProxy.$onDidActivateExtension(extensionDescription.identifier, activationTimes.codeLoadingTime, activationTimes.activateCallTime, activationTimes.activateResolvedTime, reason); this._logExtensionActivationTimes(extensionDescription, reason, 'success', activationTimes); return activatedExtension; }, (err) => { this._logExtensionActivationTimes(extensionDescription, reason, 'failure'); throw err; }); } private _doActivateExtension(extensionDescription: IExtensionDescription, reason: ExtensionActivationReason): Promise { const entryPoint = this._getEntryPoint(extensionDescription); if (!entryPoint) { // Treat the extension as being empty => NOT AN ERROR CASE return Promise.resolve(new EmptyExtension(ExtensionActivationTimes.NONE)); } const activationTimesBuilder = new ExtensionActivationTimesBuilder(reason.startup); return Promise.all([ // 插件在这里被加载,得到插件里面的入口即 activate 函数 this._loadCommonJSModule(extensionDescription.identifier, joinPath(extensionDescription.extensionLocation, entryPoint), activationTimesBuilder), // 插件的 context,激活的时候作为参数传入一次 this._loadExtensionContext(extensionDescription) ]).then(values => { performance.mark(`code/extHost/willActivateExtension/${extensionDescription.identifier.value}`); // _callActivate 调用插件的 activate 函数并传入 context 参数 return AbstractExtHostExtensionService._callActivate(this._logService, extensionDescription.identifier, values[0], values[1], activationTimesBuilder); }).then((activatedExtension) => { performance.mark(`code/extHost/didActivateExtension/${extensionDescription.identifier.value}`); return activatedExtension; }); } private static _callActivate(logService: ILogService, extensionId: ExtensionIdentifier, extensionModule: IExtensionModule, context: vscode.ExtensionContext, activationTimesBuilder: ExtensionActivationTimesBuilder): Promise { // Make sure the extension's surface is not undefined extensionModule = extensionModule || { activate: undefined, deactivate: undefined }; return this._callActivateOptional(logService, extensionId, extensionModule, context, activationTimesBuilder).then((extensionExports) => { return new ActivatedExtension(false, null, activationTimesBuilder.build(), extensionModule, extensionExports, context.subscriptions); }); } private static _callActivateOptional(logService: ILogService, extensionId: ExtensionIdentifier, extensionModule: IExtensionModule, context: vscode.ExtensionContext, activationTimesBuilder: ExtensionActivationTimesBuilder): Promise { if (typeof extensionModule.activate === 'function') { try { activationTimesBuilder.activateCallStart(); logService.trace(`ExtensionService#_callActivateOptional ${extensionId.value}`); const scope = typeof global === 'object' ? global : self; // `global` is nodejs while `self` is for workers // 真正调用插件 activate 函数的地方 const activateResult: Promise = extensionModule.activate.apply(scope, [context]); activationTimesBuilder.activateCallStop(); activationTimesBuilder.activateResolveStart(); return Promise.resolve(activateResult).then((value) => { activationTimesBuilder.activateResolveStop(); return value; }); } catch (err) { return Promise.reject(err); } } else { // No activate found => the module is the extension's exports return Promise.resolve(extensionModule); } } ... // abstract method,node 和 worker 有不同实现 protected abstract _beforeAlmostReadyToRunExtensions(): Promise; protected abstract _loadCommonJSModule(extensionId: ExtensionIdentifier | null, module: URI, activationTimesBuilder: ExtensionActivationTimesBuilder): Promise; } ``` 那我们的在插件中 `import {} from 'vscode'` 的 `vscode` 从哪里来呢?秘密在 `NodeModuleRequireInterceptor` 上。上面的 `AbstractExtHostExtensionService` 是个抽象类,`_beforeAlmostReadyToRunExtensions` node 版本的实现在 `src/vs/workbench/api/node/extHostExtensionService.ts`中,会在真正加载所有插件前做一些额外的工作 ```typescript // src/vs/workbench/api/node/extHostExtensionService.ts export class ExtHostExtensionService extends AbstractExtHostExtensionService { readonly extensionRuntime = ExtensionRuntime.Node; // 在 ExtensionHostService.initialize 中调用,在可以运行插件之前做一些劫持工作 protected async _beforeAlmostReadyToRunExtensions(): Promise { // 记住这里的 extensionApiFactory,待会要用 #1# // initialize API and register actors const extensionApiFactory = this._instaService.invokeFunction(createApiFactoryAndRegisterActors); // Register Download command this._instaService.createInstance(ExtHostDownloadService); // Register CLI Server for ipc if (this._initData.remote.isRemote && this._initData.remote.authority) { const cliServer = this._instaService.createInstance(CLIServer); process.env['VSCODE_IPC_HOOK_CLI'] = cliServer.ipcHandlePath; } // 看到这个 interceptor 了吗,vscode 劫持了插件的 require,待会再看 interceptor 的实现 // Module loading tricks const interceptor = this._instaService.createInstance(NodeModuleRequireInterceptor, extensionApiFactory, this._registry); await interceptor.install(); performance.mark('code/extHost/didInitAPI'); // Do this when extension service exists, but extensions are not being activated yet. const configProvider = await this._extHostConfiguration.getConfigProvider(); await connectProxyResolver(this._extHostWorkspace, configProvider, this, this._logService, this._mainThreadTelemetryProxy, this._initData); performance.mark('code/extHost/didInitProxyResolver'); // 还劫持了 process.send,不过这里的 ipc 只用来发 log // Use IPC messages to forward console-calls, note that the console is // already patched to use`process.send()` const nativeProcessSend = process.send!; const mainThreadConsole = this._extHostContext.getProxy(MainContext.MainThreadConsole); process.send = (...args) => { if ((args as unknown[]).length === 0 || !args[0] || args[0].type !== '__$console') { return nativeProcessSend.apply(process, args); } mainThreadConsole.$logExtensionHostMessage(args[0]); return false; }; } protected _loadCommonJSModule(extensionId: ExtensionIdentifier | null, module: URI, activationTimesBuilder: ExtensionActivationTimesBuilder): Promise { if (module.scheme !== Schemas.file) { throw new Error(`Cannot load URI: '${module}', must be of file-scheme`); } let r: T | null = null; activationTimesBuilder.codeLoadingStart(); try { if (extensionId) { performance.mark(`code/extHost/willLoadExtensionCode/${extensionId.value}`); } // 为什么是 __$__nodeRequire 呢?因为 global.require 已经被劫持过一遍了 🤣 r = require.__$__nodeRequire(module.fsPath); } catch (e) { return Promise.reject(e); } finally { if (extensionId) { performance.mark(`code/extHost/didLoadExtensionCode/${extensionId.value}`); } activationTimesBuilder.codeLoadingStop(); } return Promise.resolve(r); } // 看起来还会修改插件的 process.env public async $setRemoteEnvironment(env: { [key: string]: string | null }): Promise { if (!this._initData.remote.isRemote) { return; } for (const key in env) { const value = env[key]; if (value === null) { delete process.env[key]; } else { process.env[key] = value; } } } } // 上面提到的 NodeModuleRequireInterceptor 的实现是这样的,RequireInterceptor 的 install 实际上调用的这个方法 class NodeModuleRequireInterceptor extends RequireInterceptor { protected _installInterceptor(): void { const that = this; // 劫持 module._load,require 内部使用的就是 module._load 去实际加载模块 const node_module = require.__$__nodeRequire('module'); const original = node_module._load; node_module._load = function load(request: string, parent: { filename: string; }, isMain: boolean) { // 从 alternatives 列表查找本来应该 require 的模块的替代品(可能是 polyfill 之类的) for (let alternativeModuleName of that._alternatives) { let alternative = alternativeModuleName(request); if (alternative) { request = alternative; break; } } if (!that._factories.has(request)) { return original.apply(this, arguments); } // 另外如果 factories 中有相应模块的工厂就也替换掉 return that._factories.get(request)!.load( request, URI.file(parent.filename), request => original.apply(this, [request, parent, isMain]) ); }; } } ``` 不得不说一句**野啊宝贝**,factories 实际上是在它的父类,也就是 `RequireInterceptor` 中添加的 ```typescript // src/vs/workbench/api/common/extHostRequireInterceptor.ts export abstract class RequireInterceptor { protected readonly _factories: Map; protected readonly _alternatives: ((moduleName: string) => string | undefined)[]; constructor( // 还记得之前 ExtHostExtensionService#_beforeAlmostReadyToRunExtensions 的那个 extensionApiFactory (搜索 #1#) 吗,createInstance 的时候作为参数传入 private _apiFactory: IExtensionApiFactory, private _extensionRegistry: ExtensionDescriptionRegistry, @IInstantiationService private readonly _instaService: IInstantiationService, @IExtHostConfiguration private readonly _extHostConfiguration: IExtHostConfiguration, @IExtHostExtensionService private readonly _extHostExtensionService: IExtHostExtensionService, @IExtHostInitDataService private readonly _initData: IExtHostInitDataService, @ILogService private readonly _logService: ILogService, ) { this._factories = new Map(); this._alternatives = []; } async install(): Promise { this._installInterceptor(); performance.mark('code/extHost/willWaitForConfig'); const configProvider = await this._extHostConfiguration.getConfigProvider(); performance.mark('code/extHost/didWaitForConfig'); const extensionPaths = await this._extHostExtensionService.getExtensionPathIndex(); // extensionApiFactory 变成 this._apiFactory,这里就是 require('vscode') 模块的工厂函数,每个插件的 vscode 是不一样的,应该是为了防止插件劫持 vscode api this.register(new VSCodeNodeModuleFactory(this._apiFactory, extensionPaths, this._extensionRegistry, configProvider, this._logService)); // 还会替换掉 keytar 和 open 这两个 node 模块 this.register(this._instaService.createInstance(KeytarNodeModuleFactory)); if (this._initData.remote.isRemote) { this.register(this._instaService.createInstance(OpenNodeModuleFactory, extensionPaths, this._initData.environment.appUriScheme)); } } // 由 NodeModuleRequireInterceptor 实现 protected abstract _installInterceptor(): void; public register(interceptor: INodeModuleFactory): void { if (Array.isArray(interceptor.nodeModuleName)) { for (let moduleName of interceptor.nodeModuleName) { this._factories.set(moduleName, interceptor); } } else { this._factories.set(interceptor.nodeModuleName, interceptor); } if (typeof interceptor.alternativeModuleName === 'function') { this._alternatives.push((moduleName) => { return interceptor.alternativeModuleName!(moduleName); }); } } } // 上面的 VSCodeNodeModuleFactory 的实现是这样的 class VSCodeNodeModuleFactory implements INodeModuleFactory { public readonly nodeModuleName = 'vscode'; private readonly _extApiImpl = new Map(); private _defaultApiImpl?: typeof vscode; constructor( // 接受 factory 作为参数 private readonly _apiFactory: IExtensionApiFactory, private readonly _extensionPaths: TernarySearchTree, private readonly _extensionRegistry: ExtensionDescriptionRegistry, private readonly _configProvider: ExtHostConfigProvider, private readonly _logService: ILogService, ) { } // require('vscode') 的时候实际上 load 了这里,调用 factory 获得一个新的 vscode 对象 public load(_request: string, parent: URI): any { // get extension id from filename and api for extension const ext = this._extensionPaths.findSubstr(parent.fsPath); if (ext) { let apiImpl = this._extApiImpl.get(ExtensionIdentifier.toKey(ext.identifier)); if (!apiImpl) { apiImpl = this._apiFactory(ext, this._extensionRegistry, this._configProvider); this._extApiImpl.set(ExtensionIdentifier.toKey(ext.identifier), apiImpl); } return apiImpl; } // fall back to a default implementation if (!this._defaultApiImpl) { let extensionPathsPretty = ''; this._extensionPaths.forEach((value, index) => extensionPathsPretty += `\t${index} -> ${value.identifier.value}\n`); this._logService.warn(`Could not identify extension for 'vscode' require call from ${parent.fsPath}. These are the extension path mappings: \n${extensionPathsPretty}`); this._defaultApiImpl = this._apiFactory(nullExtensionDescription, this._extensionRegistry, this._configProvider); } return this._defaultApiImpl; } } ``` 最后 #1# 的 `apiFactory` 到底是什么呢?绕了一圈回到了 `extHost.api.impl.ts` 这个文件上 ```typescript // src/vs/workbench/api/common/extHost.api.impl.ts export function createApiFactoryAndRegisterActors(accessor: ServicesAccessor): IExtensionApiFactory { // services const initData = accessor.get(IExtHostInitDataService); const extHostFileSystemInfo = accessor.get(IExtHostFileSystemInfo); const extHostConsumerFileSystem = accessor.get(IExtHostConsumerFileSystem); ... // register addressable instances rpcProtocol.set(ExtHostContext.ExtHostFileSystemInfo, extHostFileSystemInfo); rpcProtocol.set(ExtHostContext.ExtHostLogService, extHostLogService); rpcProtocol.set(ExtHostContext.ExtHostWorkspace, extHostWorkspace); ... // automatically create and register addressable instances const extHostDecorations = rpcProtocol.set(ExtHostContext.ExtHostDecorations, accessor.get(IExtHostDecorations)); const extHostDocumentsAndEditors = rpcProtocol.set(ExtHostContext.ExtHostDocumentsAndEditors, accessor.get(IExtHostDocumentsAndEditors)); const extHostCommands = rpcProtocol.set(ExtHostContext.ExtHostCommands, accessor.get(IExtHostCommands)); ... // manually create and register addressable instances const extHostEditorTabs = rpcProtocol.set(ExtHostContext.ExtHostEditorTabs, new ExtHostEditorTabs()); const extHostUrls = rpcProtocol.set(ExtHostContext.ExtHostUrls, new ExtHostUrls(rpcProtocol)); const extHostDocuments = rpcProtocol.set(ExtHostContext.ExtHostDocuments, new ExtHostDocuments(rpcProtocol, extHostDocumentsAndEditors)); ... // Check that no named customers are missing const expected: ProxyIdentifier[] = values(ExtHostContext); rpcProtocol.assertRegistered(expected); // Other instances const extHostBulkEdits = new ExtHostBulkEdits(rpcProtocol, extHostDocumentsAndEditors, extHostNotebook); const extHostClipboard = new ExtHostClipboard(rpcProtocol); const extHostMessageService = new ExtHostMessageService(rpcProtocol, extHostLogService); const extHostDialogs = new ExtHostDialogs(rpcProtocol); const extHostStatusBar = new ExtHostStatusBar(rpcProtocol, extHostCommands.converter); const extHostLanguages = new ExtHostLanguages(rpcProtocol, extHostDocuments); // Register API-ish commands ExtHostApiCommands.register(extHostCommands); // 这里就是那个 apiFactory 的实现了,vscode 提供的全部 api 都在里面(除了 context) // TODO vscode is injected return function (extension: IExtensionDescription, extensionRegistry: ExtensionDescriptionRegistry, configProvider: ExtHostConfigProvider): typeof vscode { const authentication: typeof vscode.authentication = { getSession(providerId: string, scopes: string[], options?: vscode.AuthenticationGetSessionOptions) { return extHostAuthentication.getSession(extension, providerId, scopes, options as any); }, get onDidChangeSessions(): Event { return extHostAuthentication.onDidChangeSessions; }, registerAuthenticationProvider(id: string, label: string, provider: vscode.AuthenticationProvider, options?: vscode.AuthenticationProviderOptions): vscode.Disposable { checkProposedApiEnabled(extension); return extHostAuthentication.registerAuthenticationProvider(id, label, provider, options); }, ... }; // namespace: commands const commands: typeof vscode.commands = {...}; // namespace: env const env: typeof vscode.env = {...}; ... return { version: initData.version, // namespaces authentication, commands, comments, debug, env, extensions, languages, notebook, scm, tasks, test, window, workspace, // types Breakpoint: extHostTypes.Breakpoint, CallHierarchyIncomingCall: extHostTypes.CallHierarchyIncomingCall, CallHierarchyItem: extHostTypes.CallHierarchyItem, ... // proposed api types get RemoteAuthorityResolverError() { // checkProposedApiEnabled(extension); return extHostTypes.RemoteAuthorityResolverError; }, get ResolvedAuthority() { // checkProposedApiEnabled(extension); return extHostTypes.ResolvedAuthority; }, get SourceControlInputBoxValidationType() { // checkProposedApiEnabled(extension); return extHostTypes.SourceControlInputBoxValidationType; }, ... }; }; } ``` 至此我们知道了关于插件的一些事情 - vscode 插件全都运行在一个进程上,全局变量都是共享的(所以 `vscode.commands.executeCommand()` 可以传任意参数 - vscode 对插件用的全局变量和 node 模块做了劫持,但是既然在同一个进程上我们可以再劫持一层;要脱离 vscode 的劫持大概只能 fork 一个新的 node 进程 - 每个插件的 vscode 和 context 实例都是独立的,并且只使用 node API 的话应该是很难拿到 rpc 协议各个 actor 的 rpcId 的 在快看完的时候发现了 [API注入机制及插件启动流程_VSCode插件开发笔记2](http://www.ayqy.net/blog/api%E6%B3%A8%E5%85%A5%E6%9C%BA%E5%88%B6%E5%8F%8A%E6%8F%92%E4%BB%B6%E5%90%AF%E5%8A%A8%E6%B5%81%E7%A8%8B_vscode%E6%8F%92%E4%BB%B6%E5%BC%80%E5%8F%91%E7%AC%94%E8%AE%B02/) 这篇文章,写的还蛮好的。文章的 vscode 版本是 1.19.3,跟现在的版本差了很大,加载的部分几乎被重构了,extensionHost 与 main 的通信已经变成了 RPC + IPC (仅传递插件的 console 到主进程 console),但还是值得参考的。 最后,新年快乐 🎉 ###### 1 当然还有 mixed 类型的,不一定都是 json ###### 2 除非每个版本都根据表生成一个 rpc 协议出来 标签: vscode, vscode 插件, 黑魔法
完全看不懂,但是你又变强了