Atomese SIMD (OpenCL/CUDA) Interfaces

Experimental effort to enable I/O between Atomese and SIMD compute resources. The primary target is commodity GPUs, running either OpenCL or CUDA. The current prototype accesses the hardware at a low level, working with individual vectors. High-level API's, such as commonly used in deep learning, are not yet supported in this interface layer.

The Experiment

The experiment, as it is currently evolving, is to understand graph rewriting between "natural" descriptions of compute systems. For example, the "natural" description of a deep learning transformer is as a kind of wiring diagram, indicating where data flows. By contrast, the "natural" description of GPU compute kernels are C/C++ function prototypes. How can one perform the rewrite from the high-level description to the low-level description? Normally, this is done "brute force": a human programmer sits down and writes a large pile of GPU code, creating systems such as PyTorch or TensorFlow. The experiment here is to automate the conversion from high-level to low-level descriptions.

Such automation resembles what compilers do. For example, a C++ compiler accepts C++ code and issues machine assembly code. A Java compiler accepts Java and issues Java bytecode. The goal here is to accept high-level functional descriptions, such as that of a transformer, and to generate low-level invocations of GPU kernels. However, the project here is not meant to just be a "compiler" per se; if that is what one really wanted, one could just brute-force write one (or use PyTorch, TensorFlow, ...) Instead, this is meant to be an exploration of graph rewriting in general, and it uses the GPU target as a realistic and suitably complicated example system.

A different way of thinking of this experiment is as an exploration of the agency of sensori-motor systems. The GPU is a "thing out there" that can be manipulated by an agent to "do things". How does an agent perceive the "thing out there", and "understand" it's properties? How can the agent control the "thing out there", and perform actions on it? To exert motor control on that "thing out there"? How does the agent perceive the results of those motor actions?

In the present experiment, the "agent" is is a collection of graphs (properly, hypergraphs), represented using Atomese, and "living" in the OpenCog AtomSpace. The subjective inner form of the agent is a world-model, consisting of abstractions derived from sensory perceptions of the external world, and the knowledge of a set of motor actions that can be performed to alter the state of the external world. The agent itself is represented in Atomese; the GPU is a part of the external world, to be manipulated.

An important part of the challenge is understanding how assembly, and specifically, how self-assembly works. That is, the agent is meant to be a collection of pieces-parts, a collection of transformation rules. These rules are to be assembled to "get things done". The assembly is meant to be self-assembly, and is presumed to live at the edge of a critical phase transition. The phrase "critical phase transition" is meant to invoke the conventional idea of a critical phase transition, such as that seen in Per Bak's critical sandpile. The difference here is that the dynamical system consists of a collection of rewrite rules, instead of abstract sand-grains.

The above description might feel like excessive anthropomorphising of a mechanical system. But that's kind of the point: to force the issue, and explore what happens, when an agentic viewpoint is explicitly forced, and given the attributes of a dynamical system.

External Subsystems

Atomese is the interface language for the OpenCog AtomSpace hypergraph database. It has a variety of different ways of talking to external subsystems. These were developed to solve various practical application issues that arose in the use of Atomese. These subsystem interfaces are "well designed", in that they solve the particular issue that arose. Human programmers have used them to construct various kinds of systems. However, these interfaces lack what is needed for a dynamical, sensori-motor, agential point of view.

These external subsystem interfaces include:

• The GroundedSchemaNode allows external python, scheme and shared-library functions to be called, passing arguments encoded as Atoms.
• The StorageNode allows Atoms to be sent to and received from various locations, including Internet hosts (using CogStorageNode), disk drives (using RocksStorageNode), databases (using PostgresStorageNode), files (using FileStorageNode) and more.
• The SQL Bridge allows SQL Tables to be mapped into AtomSpace structures, so that updates to one are reflected as updates to the other.
• Obsolete gateways to ROS, the Robot Operating System, to Minecraft (via MineRL & Malmo), to Unity, the game engine, and more. These can be found in old, archived github repos, scrolling down to the oldest repos having no activity. There's also a collection of youtube videos showing these in operation; see e.g. the Virtual AI Dog demo, and, of course, the Hanson Robotics Sophia demos.

The above "work", and provide Atomese interfaces to the described systems. However, they all lack an Atomese interface description, that is, the needed IDL that would allow for reflection and introspection (in the conventional software sense of "reflection" and "introspection"). Thus, they cannot be targets of Atomese graph rewrites, because the target description is not available in machine-readable form.

Several efforts are underway to provide and understand such reflective, introspective interface definition systems. These include:

• The Sensory system, which is an experimental effort to understand the generic mathematical theory of interfacing Atomese to arbitrary unknown sensory devices, and to able to use them to obtain streams of data. This includes the abstract definition of a "motor", which is a device that can cause changes to the external world (such as movement or manipulation of objects).
• The Motor system, which attempts to define a simpler, more practical and mundane way of using Atomese to work with external devices.

This project uses the Sensory system to provide the underlying framework.

Status

Version 0.1.0 -- Basic proof-of-concept, showing how to use Atomese to open a connection to an OpenCL compute device (i.e. a GPU), load and invoke GPU compute kernels, and have those kernels work with floating-point vector data that is moved between the AtomSpace and the GPU.

New in this version: Atomese interface descriptions are now generated for OpenCL kernel interfaces. This should allow introspection of the interfaces, and their manipulation in Atomese. We'll see how that goes. Some problems are already visible. But some groundwork is there.

The demo is minimal, but it works. Tested on both AMD Radeon R9 and Nvidia RTX cards.

Overview

The directory layout follows the conventional AtomSpace standards.

Notable content:

• The examples directory contains a working example of Atomese interacting with a GPU.
• The scaffolding directory contains some bring-up code and several hello-world examples.
• Design Notes contains some raw ideas on how the system should be (and was) designed.
• The types directory contains definitions for some OpenCL Atom types.
• The atoms directory contains implementations for those Atom types.

HOWTO

Steps:

• Get some OpenCL GPU hardware. The demo should work on anything and has been tested on a Radeon graphics card and an Nvidia card.
• Install clinfo.
• For AMD devices, install mesa-opencl-icd and opencl-headers and opencl-clhpp-headers and ocl-icd-opencl-dev Maybe more; depends on your distro and hardware.
• For Nvidia devices, install cuda-opencl-dev Maybe more; depends on your distro and hardware.
• Optional: Install clang-14 and llvm-spirv-14 Demos can use "offline-compiled" (pre-built) kernels.
• User must have permission to write the device(s) located in /dev/dri; typically /dev/dri/renderD128 or similar.

    sudo usermod -a -G video <user_id>
    sudo usermod -a -G render <user_id>

• Build and install cogutils, the AtomSpace, sensory and then the code here. This uses the same build style as all other OpenCog projects: mkdir build; cd build; cmake ..; make; sudo make install
• Run unit tests: make check. If the unit tests fail, that's probably because they could not find and GPU hardware. See below.
• Look over the examples. Run them cd examples; guile -s atomese-kernel.scm

If unit tests fail

The scaffolding directory contains code that is "pure" OpenCL, and does not have any Atomese in it. It just provides some basic OpenCL examples. Runs on both AMD and Nvidia hardware.

Make sure the software isn't insane, by running the opencog/opencl/scaffolding/show-ocl-hw executable from the build directory. It will print a short hardware listing that is a subset of what the clinfo command lists. If it doesn't work, that is probably because it's too stupid to find your hardware. Read the source, Luke.

Make sure you can talk to the hardware, by running the opencog/opencl/scaffolding/run-hello-world executable from the build directory. It should print >>This is only a test<< if the code ran on the GPUs. It will work only if there is a copy of hello.cl in whatever directory that you are running run-hello-world from.

The opencog/opencl/scaffolding/run-vec-mult executable is similar to above; it performs a simple vector multiply.

The run-flow-vec executable is a rework of above, to more clearly define and prototype the distinct steps needed to flow data.

---