Along the right side of the dialog box are all the presets and controls you need for adjusting the image optimization. All the resets can be further adjusted using the various drop-down lists and controls, and then the combined settings can be saved as a preset for future use.
All the settings are explained in detail in the Photoshop Help Center (Topic > Preparing graphics for the Web > Web graphics formats and options), so I will not repeat them here. However, I would like to encourage you to read over the optimization options for JPEG, GIF, and PNG. As you read, play with the various settings, and compare file sizes and previews to get a feel for how optimization affects your image sizes.
The small arrow to the right of the settings opens the Optimize menu where you'll find options to save and delete custom settings, optimize an image to a file size you specify, repopulate previews, and work with slices.