d3.js visibility-zone calculations

i want to figure out how to properly calculate vizibility zone and draw it using d3.geo projections. visibility zone in my case is optical camera frustum

current method is wrong, but gives me approximate result

for now, i have a two plots, one in gnomonic projection:

// pixels in one degree in gnomonic projection chart with scale 1500
var px = 26.8;

var w = px * viewport.width;
var h = px * viewport.height;

d3.geoGnomonic()
  .translate([w / 2, h / 2])
  .scale(1500)

on gnomonic plot i've placed points by its border, then reproject these points using d3.projection.invert method and used resulting angles on mercator projection plot to draw areas(like here), with following results:

viewport here is a size of frustum in angles

i want to figure out what is wrong in my scenario..

from d3.js visibility-zone calculations