1// Copyright 2021-2023 The Khronos Group, Inc. 2// 3// SPDX-License-Identifier: CC-BY-4.0 4 5= VK_KHR_cooperative_matrix 6:toc: left 7:refpage: https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/ 8:sectnums: 9 10This document proposes adding support for so-called cooperative matrix 11operations that enables multiple shader invocations to cooperatively and 12efficiently perform matrix multiplications. 13 14== Problem Statement 15 16A growing number of GPU applications are making use of matrix multiplication 17operations. Modern GPU HW can take advantage of cross-invocation communication 18channels or other hardware facilities to implement matrix multiplications 19operations more efficiently but there is currently no suitable standard 20SPIR-V/API mechanism to expose these features to applications or libraries. 21 22== Solution Space 23 24Applications or libraries can use subgroup primitives to write more efficient 25matrix multiplication kernels but, while technically possible on some hardware, 26this approach often does not make it possible to write optimal kernels and 27requires applications to have a lot of device-specific knowledge. 28 29NVIDIA exposed with VK_NV_cooperative_matrix a new set of abstractions for such 30cooperative matrix operations. These include cooperative load and store 31instructions, a matrix multiplication-addition instruction as well a limited 32support for element-wise operations on these matrices. Since the release of 33that extension, a growing body of evidence in the form of discussions and 34other similar vendor extensions suggests that this approach is suitable for 35a wide variety of devices and applications and is thus a good candidate for 36standardisation. 37 38== Proposal 39 40Work towards a standard extension that exposes abstractions similar as those 41released under VK_NV_cooperative_matrix. 42 43== Examples 44 45See specifications and presentations for VK_NV_cooperative_matrix. 46 47== Issues 48 49None. 50 51