The outer wheel bearing has to come out along with the large washer. Once those are out, you "should" be able to remove the drum. If not, you might have to remove the small rubber plug from the backing plate and spin your automatic adjusting wheel in to get the drums off. Once that is done, a BFH might help to remove some of the rust built up on the edge of the drum possibly preventing the drum from coming off easily.
As long as you have the bearing retaining nut off, I would try tapping around the edge of the drum to try to drive the drum off the spindle. Usually, though, a few taps on the side of the drum is enough to get the bearing to pop loose.