Posts

Note on Denoising Diffusion Probabilistic Models

I've recently discovered a fantastic online course titled " TinyML and Efficient Deep Learning Computing " taught by  Prof. Song Han  at MIT. This course delves into the latest advancements in large language models and generative AI. While   Lecture 16  provides a comprehensive overview on diffusion models and their recent generalizations, it skips some mathematical details regarding  Denoising Diffusion Probabilistic Models  (DDPM).  This post serves as my notes on these skipped mathematical details from the lecture. Especially,  We provide a simplified and much more transparent derivation on the training loss than the one presented in the  DDPM paper .  We show that the dropped $L_T$ term in the  DDPM paper  should not appear at all if we start with the correct loss.  No special treatment is needed for the $L_0$ term in the  DDPM paper , i.e. $L_{t-1}$ is applicable for $t=1$ as well.  Forward diffusion process The forward diffusion process is to gradually add white noi

BOXX ETF

This post is for knowledge sharing only. It is not intended to be investment or tax advice. BOXX ETF  is a new financial product that offers nearly identical returns and risks to treasury bills, yet it may be more tax-efficient:  You only have to pay capital gain tax on the shares you sell, not on all the savings you have. If you hold onto the shares you sell for over a year, you'll pay the long-term capital gains tax, which is lower than the tax on both ordinary interest and treasury interest. Sell shares within a year of buying them, and you'll pay short-term capital gains tax. It's the same rate as the tax rate of ordinary income but higher than the tax rate of treasury interest. Heavy tax on interest Every family should maintain some cash reserve as family emergency fund. The headache is that the interest generated from such cash reserve is taxed as ordinary income each year. For high income earners in California, The tax rate on interest can be as high as 51% includin

More on Entropy

I reviewed some basics of thermodynamics as well as the Boltzmann entropy in this post . Here, I continue to review the Gibbs entropy, the information entropy as well as the cross-entropy that is widely used in machine learning. In the modern view, entropy is a measure of the ignorance of an observer on the system. Boltzmann Entropy Physics students start to learn the concept of entropy from the Carnot engine and Boltzmann entropy. The Boltzmann entropy applies to a single system with the fixed energy: \begin{equation}S=k_B\log \Omega\,,\tag{1}\end{equation}where $k_B$ is the  Boltzmann constant  and $\Omega$ is the number of microstates corresponding to a system's macrostate. Boltzmann also derived another form of entropy that is very close to the modern form in mathematics. Consider a single system consisting of $N$ particles, we partition $N$ particles into $n$ groups. Let $N_i$ be the number of particles in the group $i$, then the number of microstates is \begin{equation}\Omeg

How to offset W2 tax

This post is for knowledge sharing only. It is not intended to be tax or investment advice. 1. Contribute to pre-tax 401K and HSA As well known, pre-tax 401K and health saving account (HSA) are the only a few tax shelters for W2 workers.  2. Tax-loss harvesting If your total capital losses exceed your capital gains in a tax year, you can deduct up to \$3,000 (married filing jointly) from your ordinary income. If your losses are more than $3,000, you can carry over the remaining amount to future years, continuing to offset your income until all the losses are used up. A popular method to generate these deductible losses is  tax-loss harvesting . This involves selling a security at a loss to offset gains or reduce ordinary income, and optionally purchasing a similar, but not "substantially identical", security to avoid missing out on the market. However, be aware of the  wash-sale rule , which disallows claiming a loss if you buy the same or a substantially identical security

Revocable Living Trust

This post is for knowledge sharing only. It is not intended to be tax or legal advice. I used to believe that trusts were only for the wealthy. But recently, I learned that there is at least one trust that ordinary families, especially those living in California, should establish for seamless family wealth inheritance: a revocable living trust. One may need to find a lawyer on the estate planning to discuss and set up a revocable living trust. But it is still good to learn some background knowledge before talking to a lawyer.   Why revocable living trust A revocable living trust, as implied by its name, allows the grantor (who created the trust) to make changes or completely revoke the trust while they are still alive. This means the grantor maintains full control over the assets placed within the trust. As a result, assets held in a revocable living trust are NOT protected from creditors. Avoid Probate The primary reason to establish a revocable living trust is to avoid probate . Pro

ETFs for investments

 This post is for knowledge sharing only. It is not intended to be investment or tax advice. Highlight There are a lot of ETF s on the market. Personally, I only prefer the following two ETFs for long term investments: VOO  (or IVV ): Both VOO and IVV passively track the well known S&P 500 index . The expense ratios of VOO and IVV are the same: 0.03%. SPY also tracks S&P 500 index but with a higher expense ratio: 0.0945%. It is more suitable for trading rather than buying and holding. QQQM (or QQQ ): Both QQQM and QQQ passively track the Nasdaq 100 index . They are almost identical. QQQM is a newer ETF launched very recently for buying and holding. The expense ratio of QQQM is 0.15% while the expense ratio QQQ is 0.2%. Notes on QQQM (or QQQ): Although QQQM concentrates heavily on tech companies, fundamentally it is NOT a  sector ETF . For example, in the top 10 holdings of QQQM as of January 2024, we see the name of Costco Wholesale that belongs to the consumer discretionary

NO target date funds

This post is for knowledge sharing only. It is not intended to be investment or tax advice. If one does not manually select or change investments,  the default investment options in most 401K plans are usually some target data funds similar to  Vanguard Target Retirement Funds . Such funds are dynamically mixed with stocks and bonds, and the asset allocation gradually weights more to bonds from stocks. My personal opinion is very simple: NO target date funds.  Instead, choose a low-cost S&P 500 index fund in the 401K plan. Here are the main reasons: The fees of target date funds are high. For example, the expense ratio of  Vanguard Target Retirement Funds  is 0.08%. Some target funds are even more expensive. In contrast, the expense ratio of  Vanguard 500 Index Fund Institutional Select Shares is only 0.01%. The long term performance is worse than the S&P 500 index. This is not surprising because of the existence of bonds in the target date funds. The fund may also include int