Officially, USN O-4's are not Field Grade Officers, therefore they are JO's. (Field Grade gets scrambled eggs on their combo covers)
However, in a squadron environment the Department Heads are not considered part of the JO's.
In the typical squadron the O-3 and below make up the JOPA.
In aviation some CO's have mandated that the O-3 and below address O-4's as "LCDR so-and-so", others have allowed O-4's to be addressed by call-signs, some have not cared.
One a ship (SWO world), an O-3 and below will generally call an O-4 by "LCDR so-and-so" or by his position on the ship (e.g. XO, CSO, etc)
I think subs still have an O-4 XO, therefore he'll be called XO.
If you get outside the squadron environment then it sort of depends on where you are; at a COCOM, O-5 and below are what you could consider the JO's.
I went to a "Senior Officer" meeting once out at C5F; a Marine Major walked into the room and there were a lot of Navy O-5's who were just looking at him like he had three heads. The Marine explained that Senior Officer translated to Field Grade, therefore, he should be included since he's Field Grade. He ended up staying for the meeting.