Boats from the nearby Ahoushat First Nation arrived first on the scene, said aboriginal Councilor Tom Campbell. He was on the waterfront and watched as rescuers brought several survivors ashore.
"Their looks tell the whole story," he said by phone from Tofino. "You can't describe looks on people that are lost. They look totally lost - shocked and lost."
Local fisherman Lance Desilets arrived on the scene as last-known survivors were ferried to shore, CBS News correspondent John Blackstone reports.
"The Coast Guard asked us to circle around the area to see if we could find any more survivors or bodies floating in the water," Desilets said.
Desilets said the boat may have been in a rocky area that's home to a sea lion colony, Blackstone reported.
The boat, the 65-foot Leviathan II, was partially submerged 8 nautical miles west of Tofino.
"As of right now the case has been turned over to the RCMP and it's a missing person file," James said. "Fortunately, we did manage to rescue 21 people."
Tofino's mayor described the town's mood as tense but commended residents for their quick aid in the rescue effort.
"Everybody's heart is just breaking for what's going on here and wanting to be as helpful as possible," Josie Osborne said in a telephone interview late Sunday.
John Forde, who runs The Whale Centre, another whale watching operation, responded to the call for help and was told the search was for four or five missing people.
The ship that went down was operated by Jamie's Whaling Station, Forde said. He said he had no idea how it could've sunk.
"Over the course of a season and years we take out thousands and thousands of people on these trips in conditions similar today," Forde said. "I have no idea what the issue was or what actually happened."
Forde said Jamie's Whaling Station was one of the first of its kind off Vancouver Island and had been around for many years.
Tofino fishing guide Lance Desilets said at least 12 rescue boats were already out on the water when he arrived on the scene after responding to the call for help.
"I saw a lot of personal belongings, a long diesel slick and the top 10 feet of the Leviathan II sticking out of the water," Desilets said. "It's a sad day for our community."
It wasn't the first fatal accident on the whale watching company's record. In 1998 one of its vessels capsized during an excursion, sending all four people on board into the water. The operator and a passenger died.
Many of the survivors were taken to Tofino General Hospital and some were already discharged Sunday night, said Valerie Wilson, a spokeswoman for the Vancouver Island Health Authority.
Canada's Transportation Safety Board said it was investigating the boat's sinking.
Joe Martin, a member of the Tal-o-qui-aht tribe, was near the dock when rescue boats went out.
The ship was on the far side of Vargas Island in Clayoquot Sound, an area that Martin said can get really rough, but was not on Sunday.
"It wasn't even blowing hard," he said. "This is the largest boat in Tofino and I was really surprised that it went down."