Police say James Hennessy, 58, of Winnie Street was making anonymous calls to his neighbors. He was allegedly using racial slurs, even threatening the two family's lives.
"Threatening my nephew saying they were going to kidnap him basically they wanted the black child out, racist remarks," said Mr. Martinez, one of the victims.
The victims say they do not know him. But he lived within a block from them.
"For a while we didn't let him out we were in fear something may occur," said Martinez.
He says that part of the fear came from not knowing who was on the other end. The numbers and voices changed, with each call.
"The caller ID coming to those victims changed. One came back to St. Peter's, another came back to Andy's Sports Bar, another came back to the Ku Klux Klan in Arkansas," said James Miller, the Albany Police Department spokesperson.
Police say those three places were not actually where the calls were coming from.
Hennessy was allegedly using Trac Phones and the website BluffMyCall.com to disguise himself by typing in those other location's numbers, while the website changed his voice.
Police used that technology to catch him.
"They pieced together calls from the Trac Phone to the IP address of the online service and put together the times and dates," said Miller.
There was no answer at Hennessy's home Friday night. Police say he faces felony harassment charges.
"I just hope it gets resolved. We don't want to live like this. No one has to," said Martinez.
Police say that Hennessy told them he is a lawyer and a retired New York City Police Officer, after serving for 20 years.
This was a joint investigation by the Albany Police, the Albany County District Attorney and the FBI. The investigation is ongoing.