University of Illinois at Urbana-Champaign

Next-Generation Compilers and Architectures for Computation Acceleration with Energy Efficiency

• About The Project
• People
• Publication

About The Project

A. Introduction

One of ADSC’s central goals at the beginning is to enable Immersive Virtual Reality. Such applications can play essential roles to facilitate humans to interact with the digital world in a seamless and natural manner, as envisioned by the Human Sixth Sense Program (HSSP) in ADSC. However, the majority of the related Digital Signal Processing (DSP) algorithms have been previously implemented in software running on CPUs, which might not be fast enough or the most energy efficient. This was the motivation for PI Chen to propose using either FPGAs or GPUs to speed up the kernel computations of the relevant DSP algorithms three year ago. Through a collaborative effort within ADSC, we demonstrated that FPGAs and GPUs could provide a significant amount of speedup and improvement for energy efficiency. The initial achievements from previous project illustrate a great potential for GPUs and FPGAs to tremendously speed up multimedia applications with high energy efficiency.

However, it is also apparent that exploiting the computing resources in these accelerators for higher performance and lower energy is in most cases far away from being a push-button process. Programming models for GPUs and FPGAs are completely isolated and disjoint. Efficient programming of both architectures requires widely different skills. To get the best performance from different platforms, programmers have to come up with specialized implementations targeting each architecture. Our initial FCUDA effort [1-4] tackled these challenges from the head on. We developed appropriate translation systems to compile parallel programs in CUDA (a GPU language) into customized parallel execution on FPGAs. Such a flow offers several major advantages. First, it allows application developers to use a common programming model for both GPUs and FPGAs in a heterogeneous platform (such as [16]), which simplifies application development. Second, due to CUDA’s wide adoption and popularity, a large body of existing applications written in CUDA would be available for FPGA acceleration. Third, applications that do not gain much benefit from GPU execution can be retargeted to FPGAs for potentially better acceleration. Fourth, it will enable very rapid and easy evaluation of a large amount of parallel algorithms. FPGAs traditionally have a complex design flow and have required programming using low-level hardware languages. This is starting to change with research in high-level synthesis (HLS) tools that map higher level programming models onto FPGAs, thereby improving design productivity tremendously. Driven by the same motivation, we also started to develop HLS techniques that drastically improve the efficiency of FPGA programming with high design quality [8-12].

GPUs have gained importance as one of the primary computation units used for acceleration of highly data-parallel computation functions. In the desktop and portable computer markets, GPUs are now ubiquitous; every system has a graphics processor for display, and these GPUs also support general purpose computations (GPGPU) through the CUDA and OpenCL languages. However, although these languages are widely used, optimizing program implementations is not a trivial task. GPGPU optimization requires in-depth knowledge of the GPU’s architecture. In the past three years, we have worked on new GPU compiler techniques for higher performance considering unique GPU architectural features, including control flow divergence optimization [5], register and thread co-optimization [6], and GPU kernel scheduling [7]. We also explored GPU acceleration targeting a specific application relevant to ADSC’s central research theme [13][14]. However, we have not worked on power and energy optimization for GPUs mainly due to a lack of sufficiently accurate GPU power/energy estimation tools and insufficient knowledge of the internal characteristics of the GPU architecture. Fortunately, the GPGPU-sim tool was recently extended with power and energy modeling features to become GPUWattch, the first cycle-accurate power and energy modeling software for GPU architectures [15], which would pave the way for our new power/energy-centric GPU research.

B. Project Goals

We group the goals of this project into two big research lines.

B.1. FPGA and HLS

For this line of research, we set two new thrusts based on the past success:

1. A complete and mature CUDA (and OpenCL) to FPGA compilation flow supporting various types of FPGA on-chip communication scenarios, thus achieving scalable high-performance FPGA solutions;
2. A continued development of the new HLS engine that would outperform the best state-of-the-art algorithms and solutions.

Our eventual target is to develop a complete open source academic design flow consisting of two components: CUDA/OpenCL to C code translation and an HLS flow that compile C/C++ code to RTL (register-transfer level) implementation for FPGA. The C/C++ to RTL flow by itself would be a standalone package. Such a path would offer modular solutions that provide important and unique flexibility for input language choices and generate FPGA solutions serving various purposes – whether they are comparing GPU/FPGA performance/energy, leveraging common languages to program both GPUs and FPGAs, or simply converting an existing C/C++ software solution to FPGA. After releasing such an open source design flow, we would also put together a commercial license so interested companies can have a chance to use it for their design purposes with a license fee.

B.2. GPU

We set two new thrusts in this line of research:

1. Advanced GPU compiler and architecture study for low energy applications;
2. Novel implementations of important multimedia applications onto GPUs.

For the first thrust, we will carry out a wide range of scheduler, compiler and architectural research to optimize power and energy consumption and properly consider tradeoffs among performance, instantaneous power consumption, and total energy consumption. For the second thrust, we will mainly focus on the applications in the embedded system projects of I2R (the Institute for Infocomm Research), especially the social robotic program, which fits into ADSC’s HSSP theme as well. There will be nice synergy between the two thrusts above. For thrust 2, in an ongoing collaboration with I2R, we will develop computer vision, audio direction finding, and intelligence algorithms for implementation in an embedded environment. With this real-world driving application, we can effectively demonstrate the impact of the fundamental research of thrust 1.

C. Research Approach and Methodology

C.1. CUDA/OpenCL to FPGA Compilation

CUDA and OpenCL are the two programming models targeting GPU architectures. CUDA targets NVIDIA GPUs while OpenCL is an open standard. We already developed a CUDA-to-FPGA compilation flow [1-4] and generated quite some interest in the research community (initial works [1][2] together have been cited more than 60 times so far). Recently, OpenCL is gaining popularity being an open and platform-independent standard. In this project, we will adapt our existing FCUDA flow to map OpenCL kernels onto FPGA hardware. Figure 1 demonstrates the new flow.

C.2. New HLS Engine

In this thrust, we will address several challenges in HLS:

1. Achieve timing closure to meet the designer’s timing constraints or deliver maximal performance.
2. Meet the resource usage constraints or minimize the resource usage to maximize the spatial parallelism.
3. Exploit the fine-grained parallelism provided by FPGA.

C.3. Advanced GPU Compiler and Architecture

We propose three subtasks in this thrust that would cooperate to achieve low power/energy-centric GPU solutions.

C.3.a. Scheduler level optimization

GPU kernels are designed for highly data-parallel computation. However, not all kernels or kernel calls require full use of the GPU: memory bound kernels may need only a subset of GPU resources to saturate performance, and some kernels may only require a subset of resources for the entire call. In our prior work [7], we demonstrated modeling and scheduling of resources for concurrent kernel execution that improves total runtime by scheduling memory-bound and computation-bound kernel concurrently. In large scale systems, the system may be sharing hundreds or thousands of GPUs among tens of thousands of simultaneous applications, we propose to:

1. Develop decentralized queuing and workload distribution methods that distribute GPU kernels into independent queues that can efficiently schedule kernels concurrently;
2. Develop methods for workload distribution that can properly order kernel dependencies while maximizing inter-kernel parallelism and minimizing communication overhead between dependent kernel instances;
3. Automate kernel splitting and merging to improve workload distribution granularity and improve expressed parallelism.

C.3.b. Compiler level optimization

Based on our prior work for joint register and thread optimization [6] and elimination of control flow divergence [5], we have identified a few new concrete directions:

1. Memory overlap optimization through use of the graph coloring techniques we developed for our FCUDA tool [3].
2. Memory space optimization through analyzing the use patterns of different memory structures in order to automatically move data structures into the texture and scratchpad memories that are typically under-utilized. Automating this remapping improves memory bandwidth utilization, data locality and lowers power and energy consumption through localization of communication.
3. Memory coalescing optimization which ensures efficient use of the memory bandwidth through appropriately grouping memory requests together into a single transaction.

C.3.c. Architectural optimization

There is significant opportunity to improve the power and energy consumption of GPU architectures. Current GPU architectures support multiple voltage and frequency settings, which can be used for traditional DVFS. However, this is only at the level of the entire GPU – when multiple kernels are sharing GPU resources, we cannot individually scale voltage and frequency of portions of the GPU architecture. The granularity of DVFS is an important design point – fine-grained DVFS gives more design and use flexibility, but increases area overhead to implement independent power and clock supply circuitry as well as multi-clock domain crossing circuitry. Coarse-grained DVFS reduces overheads, but gives less flexibility in scaling performance and power/energy consumption. Clock gating and power gating are widely used low-level architectural optimizations for reducing power consumption by turning off unused portions of the GPU. However, like DVFS, the granularity of this circuitry is an important design consideration as well. Furthermore, clock gating and power gating require sophisticated control to turn portions of the circuitry on and off without affecting kernel correctness or performance. We will explore these architectural design dimensions for power/energy-efficient GPU architecture design.

C.4. GPU Implementation of Multimedia Applications

We started a collaborative effort with researchers from I2R on their social robotic program. Their robot is currently connected to a PC, which is not ideal for mobility. To overcome this issue, we have done initial feasibility studies for board-level solutions. For a face detection algorithm I2R provided, we mapped their code into GPU using OpenCV (an open source and standard computer vision library) and tested the performance using their datasets (160×120 and 320×240 color videos). We obtained ~5x and ~13x speedup for respective resolutions.

Advanced Digital Sciences Center
1 Fusionopolis Way
#08-10 Connexis North Tower
Singapore 138632

Phone: +65 6591 9090 | Fax: +65 6591 9091