Pass a Stock Journal --- for adjusting the Qty and Amount -- to match with Closing (verified) Stock. Physical Stock -- as the name suggests is ONLY for matching the un-identified difference in the Stock quantity, without affecting the Amount.
While passing a stock journal --- Dr/Cr the appropriate ledger ---- e.g. - Difference in Physical Stock OR Loss in Transit OR Manufacturing wastage etc ........