Historian tells of frustration after four book events were cancelled over fears of far right violence