OpenGL stands for Open Graphics Language.
It is a rather low-level interface for programmers to draw polygons into the computer graphics hardware. However, the abstraction layer it provides means it is very portable. OpenGL had little mindshare of developers a few years ago when the other graphics APIs included Microsoft's Direct3D (part of DirectX) and 3dfx's Glide when one thing changed everything. That one thing was Quake, when JohnCarmack's id software chose OpenGL. Quake, combined with a Voodoo2 graphics card and the OpenGL drivers that came with it, was irrestible.
As far as Linux is concerned, XFree86 has supported OpenGL pretty well since the 3.3.x series, and the newer 4.x series has great support. Mesa is a Free implementation of OpenGL in software. Some card manufacturers have release specifications that allow developers to write drivers to take advantage of hardware acceleration. Other manufacturers are not so cooperative, notably nVidia, and offer BinaryDrivers. See Nvidia OpenGL Configuration mini-HOWTO or our NvidiaDriverHowto for specific details and some more background to how OpenGL works with XFree86.