Parallel computing is a process in which units of work are broken up into small chunks or jobs that can be executed at the same time.
Users looking to accelerate their computations should try to make use of parallel execution. Simply running code on a "faster" machine is unlikely to make large differences in the computation time. Newer CPUs and GPUs make heavy use of multiple core architectures and programs must be tuned to take advantage of these cores. Programs that have not been customized to use multiple core systems will often run on a single core of a multicore system and see no speed advantage.
Mathematics Software Packages
Many of the major Mathematics software packages will include some form of automatic parallelization. While these features can help to speed up computations they often offer minimal benefit at best. Users must make explicit use of parallel computation functions to fully leverage the available computational resources.
Mathematica includes an array of parallel computation features and functions. Documentation on the included parallel functions can be found on the Wolfram Parallel Computing page.
MATLAB provides the Parallel Computing Toolbox to leverage multicore and GPU systems. Documentation for the toolbox can be found here. Users just getting started with MATLAB's parallel computing features should begin by looking at the parfor and batch commands.
Further examples and documentation for the Parallel Computing Toolbox can be found on the MathWorks site here.
A limited number of licenses are available for the Parallel Computing Toolbox. Users should consider making use of the MATLAB Compiler to produce standalone executables that do not require use of the toolbox license. For compiler options run mcc without arguments from any system with MATLAB installed.
Maple provides a number of built-in parallel functions and optimizations. Documentation can be found on the Maple Multithreaded Programming page.
See package specific documentation for other software packaes.
C, C++, Fortran, etc.
Most major programming languages include some support, or support through libraries, for parallel computation. Please consult the documentation for your language to identify the appropriate parallel computing functions and features.