Small glass cells filled with Rubidium vapor are promising candidates for few photon nonlinear optics and quantum information processing using Rydberg states and the dipole blockade effect. Dipole blockade is a consequence of the strong interaction between two Rydberg atoms which can detune the Rydberg atom transition off resonance (pair excitation vs. single atom excitation). As a result of dipole blockade, only one Rydberg excitation is possible within a volume characterized by the blockade radius (typically a few microns), which is determined by the laser bandwidth and the Rydberg atom interaction strength. The effect is a nonlinear one that can be used as a tool to entangle atoms. Similar to atom entanglement, an atomic vapor confined on a length scale comparable to the blockade radius can be used like quantum wells (2D), quantum wires (1D) and quantum dots (0D) e.g. to realize a single photon source. We present measurements of electromagnetically induced transparency (EIT) in Rb vapor cells with thicknesses on the order of the blockade radius. We use the observed EIT-spectra to investigate the effects of the confinement in these vapor cells. The experiments show that coherent dynamics involving Rydberg states are possible in micro cells at room temperature.